{"id":"f1b0a0fc-a828-43bc-b498-c8cde265fb40","arxiv_id":"2507.21573","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A post-pruning method uses pivoted QR decomposition to delete linearly dependent CNN filters and least-squares kernel adjustment to preserve output, gaining small extra compression without fine-tuning.","lead":"LinDeps is a post-pruning add-on that detects filters whose outputs are linear combinations of other filters in the same layer, removes them, and rewires the next layer's kernels to recover the lost signal without retraining. It claims small extra compression on top of existing pruners at unchanged accuracy, which matters when fine-tuning is too expensive.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 3's recovery derivation only covers feed-forward layer chains; ResNet residual/skip connections are never handled, so the 'guaranteed performance preservation' claim is unsubstantiated for the ResNet experiments.","rationale":"The reader's weakest_assumption lists batch stability first and residual/skip handling second. We agree with the residual concern but not with the emphasis: batch stability is an empirical generalization question that additional experiments might resolve, whereas the residual-addition issue is a structural mismatch with the derivation. Section 3's L-recovery is meaningful only when the pruned layer's output is consumed by a parameterized next layer. An elementwise addition has no weights to multiply by L, so pruning the second conv of a residual block cannot be compensated by the described mechanism; the shortcut must be projected or that layer must be skipped. The paper neither states such a restriction nor provides the missing equations. This is not a disagreement with the field; it is an internal gap between the stated method and the reported ResNet results. A small instrumentation experiment on a NORTON checkpoint would settle whether the implementation avoids the problem by skipping those layers or hits it by pruning them anyway. We therefore keep a conditional verdict rather than rejecting outright, but the condition should explicitly require the authors to describe and validate residual handling before the 'guaranteed' language is credited.","tokens_in":14907,"tokens_out":7247,"duration_ms":90676,"concrete_test":"Run LinDeps on a NORTON-pruned ResNet-56 checkpoint with tau = 0, and instrument every residual block. For the second conv of each basic block, compare the block output (including the shortcut) before and after applying LinDeps on the same 256-image calibration batch. If the method prunes any filters in that conv and the block output changes by more than machine epsilon, the Section 3 recovery mechanism does not cover residual connections. If the method refuses to prune that conv, report which layers were actually pruned, since the ResNet results then rely on an undocumented restriction.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central guarantee is that after pruning layer i, the next layer's kernels are multiplied by the recovery matrix L (Eq. 4), so that the output of layer i+1 is preserved. This argument assumes the pruned layer's output is consumed only by a parameterized next layer. In a ResNet basic or bottleneck block, the second conv's output is not consumed by another kernel; it is added elementwise to the shortcut. If LinDeps prunes filters of that conv, the main branch has C' < C channels while the shortcut still has C channels, so the addition is dimensionally invalid without a projection. If the method restricts pruning to layers whose outputs feed convolutions (e.g., only the first conv of each block), that restriction is never stated in Section 3 or in the implementation details. The paper says only that BN parameters are pruned together with a channel; it never mentions residual-addition handling. Tables 2-4 report ResNet-56/110/50 results, so the claimed 'guaranteed performance preservation' and 'new state of the art' rest on an undocumented mechanism that the equations in Section 3 do not describe.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces LinDeps, a post-pruning method that operates on top of an existing structured pruning technique. For each layer, it flattens a batch of feature maps into a matrix A, applies pivoted QR decomposition to A^T, identifies (near-)linearly dependent channels via a threshold tau, prunes those channels, and computes a least-squares recovery matrix L such that L A' approximately reconstructs A. The next layer's kernels are then reshaped through L so that the layer can consume the pruned representation without fine-tuning. Experiments combine LinDeps with APIB and NORTON on VGG-16, ResNet-56, ResNet-110, and ResNet-50 on CIFAR-10 and ImageNet, reporting additional FLOPs/parameter reductions at essentially unchanged accuracy, plus a low-resource setting without retraining where larger pruning gains are obtained with small accuracy losses. The central claim is that LinDeps provides a fine-tuning-free, performance-preserving way to remove layer-wise linear dependencies.","tokens_in":15114,"tokens_out":5034,"duration_ms":67096,"significance":"If the central claim is fully supported, LinDeps would be a useful and simple add-on: the per-layer derivation is clear, the method is generic across pruning techniques, and the authors make an honest effort to report multi-seed averages for the APIB combination and to show standalone limitations. Credit is due for presenting a concrete, falsifiable experimental protocol and for separating the performance-preserving regime from the lossy low-resource regime. However, the incremental gains in the main performance-preserving tables are small (mostly 0.16--2.82% additional FLOPs reduction), and the strongest headline gain (+30% pruning in the low-resource setup) comes with accuracy loss rather than with the claimed guarantee. The principal gap is that the theoretical derivation in Section 3 covers only feed-forward layer chains, while the experiments include ResNet architectures whose residual additions are not handled in the equations. Given that gap, the 'guaranteed performance preservation' claim is not yet substantiated for a substantial part of the experimental evaluation.","major_comments":[{"comment":"The recovery derivation assumes that the pruned layer's output is consumed only by a parameterized next convolution, but this is not true for the residual blocks used in ResNet-56/110/50. If a filter of the second convolution of a block is pruned, the main branch produces C' channels while the shortcut still produces C channels, making the elementwise addition dimensionally invalid. If LinDeps is restricted to pruning only the first convolution of each block, or to layers whose outputs are not added to a shortcut, that restriction is never stated in Section 3 or in the implementation details. The equations therefore do not substantiate the 'guaranteed performance preservation' claim for the ResNet results reported in Tables 2--4.","section":"Sec. 3, Eq. (3)--(4); Tables 2--4"},{"comment":"The title and abstract promise a guarantee, but the recovery matrix L is obtained by a least-squares solve on a single batch of 256 images and the practical pruning threshold is tau = 1e-6, so the identity L A' ≈ A is approximate even on the calibration batch and unverified on test data. The paper's 'lossless pruning' terminology should be weakened to something like 'approximately output-preserving on the calibration batch,' and the performance-preservation claim should be presented as an empirical property unless a stability argument is added.","section":"Sec. 3, Eq. (4); Implementation details"},{"comment":"Equation (1) writes an exact zero block in R, but with numerical feature maps the factorization has small nonzero entries everywhere; the thresholding in Eq. (2) then changes the effective rank, and the subsequent description of selecting C' retained rows is not fully precise. Please state explicitly that Eq. (1) is the exact-dependency idealization, and define C' as the number of diagonal entries of R11 that survive the threshold, along with the exact rule for mapping the permutation P back to channel indices to form A'.","section":"Sec. 3, Step 2; Eq. (1)--(2)"}],"minor_comments":[{"comment":"The text says 'for tau = 0' but the implementation section defines lossless pruning as tau = 1e-6; please reconcile these two statements so the reader knows which threshold produced the reported 24.9% pruning result.","section":"Sec. 4.2, 'LinDeps as a standalone pruning technique'"},{"comment":"The low-resource gains are reported as single numbers without variability or confidence intervals; adding multiple runs or error bars would strengthen the claim, especially because the gains reach 30% and the accuracy losses are up to 0.3%.","section":"Sec. 4.3, Fig. 4 and Table 5"},{"comment":"The manuscript cites the public code of APIB and NORTON but does not provide a link to the authors' own implementation or checkpoints of LinDeps; please add a code release or a detailed pseudocode listing.","section":"Reproducibility"},{"comment":"The legend in Fig. 3 is dense and hard to read because many methods overlap; consider splitting the benchmark into separate panels or using a table-style legend.","section":"Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The most convincing part of the paper is the VGG-16/CIFAR-10 experiment, where the proposed mechanism is well matched to the feed-forward architecture. The ResNet results are currently not supported by the theory in Section 3, so the main revision should either extend the derivation to residual connections or explicitly restrict and re-label the claims. The 'guaranteed' language should be softened throughout, as the numerical implementation is approximate. I would not reject the paper, because the core idea is sound and the experiments are useful, but the load-bearing gap about residual connections must be fixed before the manuscript can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is a small but genuine improvement over LDFM: after pruning dependent filters via pivoted QR, you compute a least-squares recovery matrix and fold it into the next layer's kernels. That step is new, the per-layer derivation is clean, and the math is correct for a plain feed-forward chain. The paper is also honest about the standalone limitations and takes some care with reproducibility, averaging APIB results over eight fine-tuning runs.\n\nThe empirical picture is mostly as advertised: LinDeps gives consistent, if usually small, extra compression at unchanged accuracy, and the low-resource no-fine-tuning experiments show a more substantial gain (up to 30% extra pruning with <0.3% accuracy loss). That part is a practical contribution.\n\nNow the soft spots, in descending order of severity. The big one, which the stress-test note correctly identifies, is that Section 3 only covers the case where the pruned layer's output feeds another parameterized layer. In a ResNet basic or bottleneck block, the second conv's output is added to the shortcut. If you prune filters in that conv, the channel counts no longer match and the addition breaks. The paper never explains how residual/skip connections are reconciled, yet Tables 2–4 report ResNet-56/110/50 results. The accuracies hold, so the authors must be doing something sensible, but the 'guaranteed performance preservation' claim is unsubstantiated for exactly the architectures where residual connections appear. This is the difference between a solid method and a proven one.\n\nSecond, the word 'guaranteed' oversells the math. With τ=1e-6 and a least-squares solve, preservation is approximate on the calibration batch, and test accuracy preservation is an empirical observation, not a theorem. A more careful wording would help.\n\nThird, the paper never compares against LDFM, the direct predecessor, even though LDFM is mentioned as the source of the QR step. That comparison is s needed to quantify what the recovery matrix actually buys. No code is released either, which matters given the undocumented residual handling.\n\nThese are fixable. The per-layer mechanism is sound, the experimental protocol is reasonable, and the method could be a useful tool for low-resource deployment. I would send it to peer review, but with a clear request for major revision: explain the residual handling, add the LDFM comparison, release code, and soften the guarantee language. The right reader is someone working on structured pruning or model compression without a fine-tuning budget; they will get a genuinely useful add-on once the gaps are closed.","headline":"A useful fine-tuning-free post-pruning add-on with sound per-layer math, but the 'guaranteed preservation' claim is not established for ResNets because residual connections are never addressed.","tokens_in":15668,"tokens_out":2566,"would_cite":true,"duration_ms":32542,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LinDeps finds linearly dependent filters, removes them, and rewrites the next layer's kernels so accuracy survives with no fine-tuning.","keywords":["CNNs","network pruning","post-pruning","linear dependency analysis","pivoted QR decomposition","signal recovery","filter redundancy","model compression"],"falsifier":"Use a trained VGG-16, calibrate LinDeps on one 256-image batch with τ=0, and feed a held-out batch through the original and pruned networks; if the relative output error of the first pruned layer, ||L A'_test − A_test||_2 / ||A_test||_2, is much larger on the held-out batch than on the calibration batch, the recovery is batch-specific. Repeating the whole procedure with several different calibration batches and checking whether the pruned-channel sets or test accuracies vary would directly test the stability assumption on which the claimed performance preservation rests.","tokens_in":14700,"feed_emoji":"✂️","tokens_out":9496,"duration_ms":104565,"temperature":0.7,"pith_summary":"LinDeps is a post-pruning add-on that can be placed on top of any existing pruning technique, and the paper's central claim is that it removes the redundancy that other methods leave behind. That redundancy is expressed as linear dependence between feature maps: for a layer with C channels, batch images are flattened into a matrix A, a pivoted QR decomposition exposes how many channels are truly independent, and channels that are (near-)linear combinations of the rest are pruned. A recovery matrix L, obtained by least squares so that L A' ≈ A, is then applied to the next layer's kernels, which keeps the layer's output intact on the calibration batch and, in the experiments, leaves reported top-1 accuracy unchanged without any fine-tuning. Across CIFAR-10 and ImageNet with VGG-16, ResNet-56/110, and ResNet-50, LinDeps increased FLOPs reduction by about 0.16 to 5.7 percentage points on top of APIB and NORTON, and in a no-retraining low-resource setting it added up to 30 percentage points of pruning with under 0.3% accuracy loss. The paper therefore positions LinDeps as a universal complement that makes any pruning method more aggressive at almost no cost.","feed_headline":"LinDeps prunes dependent CNN filters without retraining","feed_subtitle":"QR decomposition plus a recovery matrix lifts compression on VGG and ResNet while top-1 accuracy holds.","key_machinery":"The load-bearing object is the pair formed by pivoted QR decomposition and the recovery matrix L. Pivoted QR writes A^T P = QR, with the diagonal of R11 sorted by decreasing magnitude; values below τ(R11)11 mark channels to prune, and the permutation P maps back to original channel indices. The recovery matrix L ∈ $R^{{C×C'}}$ is the least-squares solution to L A' ≈ A, encoding each removed channel as a linear combination of survivors. Multiplying a next-layer kernel, flattened to $p^{2}$ × C, by L and reshaping to C'×p×p makes the next layer accept the pruned feature maps and emit approximately the same outputs, which is what lets many layers be pruned sequentially without fine-tuning.","core_discovery":"LinDeps's discovery is that a layer's redundant filters can be identified exactly as linear dependencies among its feature maps and removed without retraining, provided the next layer is algebraically compensated. Concretely, the feature maps of a layer over a batch of B images are flattened and stacked into a matrix A ∈ $R^{{C×(B·H·W)}}$. A pivoted QR decomposition of A^T orders the channels by importance and reveals the effective rank on the diagonal of the triangular factor R11; a threshold τ zeroes small diagonal entries, leaving C' channels. LinDeps then solves L = argmin_{L} ||L A' − A||_2, where A' holds the surviving channels, and multiplies each flattened next-layer kernel (a $p^{2}$ × C matrix) by L to obtain a C'×p×p kernel. At τ = 0 this makes the next layer's output identical on the calibration batch; at τ > 0 the reconstruction is approximate. Empirically, the paper reports that this post-pruning step raises FLOPs and parameter reduction on top of APIB and NORTON while the reported top-1 accuracy stays the same across VGG-16, ResNet-56, ResNet-110, and ResNet-50.","pith_inferences":["A natural stress test the paper does not run is seed-dependence: applying LinDeps with different 256-image calibration batches and comparing pruned-channel sets and test accuracy would reveal how much the method relies on the batch being representative.","Because the recovery step is a pure least-squares linear map, the same compensation could in principle be applied to other linear layers such as attention projections in transformers, though the paper only mentions transformer pruning as future work.","The held-out reconstruction residual ||L A'_test − A_test||_2 relative to the calibration residual could serve as a per-layer confidence score, letting practitioners skip layers where the linear model is unstable instead of using one global threshold τ."],"forward_implications":["LinDeps can be stacked on top of any current or future pruning method, so its compression gain is additive rather than an alternative to importance-based pruning.","With τ = 0 (implemented as τ = 10^{-6}) the layer output is preserved exactly on the calibration batch, making the method usable where retraining is impossible, such as test-time adaptation or on-device updates.","On VGG-16/CIFAR-10, NORTON+LinDeps reaches 96.12% FLOPs reduction and 98.66% parameter reduction while keeping 92.54% top-1 accuracy, exceeding NORTON alone at 95.58% FLOPs.","The gains transfer to ResNet-56, ResNet-110, and ResNet-50, but shrink on ImageNet, which the authors attribute to noisy feature maps in a lower-accuracy model.","On low-resource devices, LinDeps itself runs quickly (13 s on an RTX 3070, 247 s on a Raspberry Pi) and shortens inference by 21-33% after NORTON pruning."],"supporting_citations":[{"why":"Supplies the QR-decomposition-on-feature-maps idea that LinDeps's pruning step is built on.","marker":"[34]"},{"why":"Provides the pivoted QR and least-squares numerical machinery used to expose dependencies and compute the recovery matrix.","marker":"[12]"},{"why":"APIB is one of the base pruning techniques LinDeps is applied to and is the source of the reproduced VGG-16 baselines.","marker":"[13]"},{"why":"NORTON is the main base pruning technique and state-of-the-art comparison; its released checkpoints are post-pruned by LinDeps.","marker":"[35]"},{"why":"The parameter and FLOPs counting utility used to measure the reduction ratios reported in the experiments.","marker":"[38]"}],"fun_headline_variants":["LinDeps: prune dependent filters, no fine-tuning","Remove dependent CNN filters without retraining","QR-based post-pruning preserves CNN accuracy","LinDeps: zero-retrain pruning for dependent filters","Prune dependent filters, keep CNN accuracy, no retrain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that linear dependencies measured on one 256-image batch persist on unseen test data, so the least-squares recovery matrix computed on that batch keeps later layers' outputs nearly unchanged after many pruning steps, and that residual and skip connections in ResNet are reconciled even though the paper's equations do not state how.","fun_headline_variants_meta":{"raw":{"variants":["LinDeps: prune dependent filters, no fine-tuning","Remove dependent CNN filters without retraining","QR-based post-pruning preserves CNN accuracy","LinDeps: zero-retrain pruning for dependent filters","Prune dependent filters, keep CNN accuracy, no retrain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000736,"raw_usage":{"total_tokens":3357,"prompt_tokens":1080,"completion_tokens":2277,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":696,"completion_tokens_details":{"reasoning_tokens":2202}},"tokens_in":696,"tokens_out":2277,"duration_ms":21304,"temperature":1.0,"reasoning_tokens":2202,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:36:55.471004+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use a trained VGG-16, calibrate LinDeps on one 256-image batch with τ=0, and feed a held-out batch through the original and pruned networks; if the relative output error of the first pruned layer, ||L A'_test − A_test||_2 / ||A_test||_2, is much larger on the held-out batch than on the calibration batch, the recovery is batch-specific. Repeating the whole procedure with several different calibration batches and checking whether the pruned-channel sets or test accuracies vary would directly test the stability assumption on which the claimed performance preservation rests.","supporting_citations":[{"cited_title":"Network pruning using linear dependency analysis on feature maps","cited_arxiv_id":null,"evidence_quote":"Supplies the QR-decomposition-on-feature-maps idea that LinDeps's pruning step is built on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the pivoted QR and least-squares numerical machinery used to expose dependencies and compute the recovery matrix."},{"cited_title":"Automatic network pruning via Hilbert-Schmidt indepen- dence criterion lasso under information bottleneck principle","cited_arxiv_id":null,"evidence_quote":"APIB is one of the base pruning techniques LinDeps is applied to and is the source of the reproduced VGG-16 baselines."},{"cited_title":"Enhanced network compression through tensor decomposi- tions and pruning","cited_arxiv_id":null,"evidence_quote":"NORTON is the main base pruning technique and state-of-the-art comparison; its released checkpoints are post-pruned by LinDeps."},{"cited_title":"ptflops: a flops counting tool for neu- ral networks in pytorch framework","cited_arxiv_id":null,"evidence_quote":"The parameter and FLOPs counting utility used to measure the reduction ratios reported in the experiments."}],"review_version":1}