{"id":"d1580b9c-e4bc-4cc5-80b0-9e4978e83d99","arxiv_id":"2608.08374","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A learned signed projection along a local neighbourhood direction produces small mean C-index and classification gains in pathology transformers, with most gains within one standard deviation.","lead":"Gated SRP is a small module that corrects pathology transformer attention by subtracting a learned amount of each patch token's alignment with its local tissue neighbourhood. Across five cancer survival and five classification tasks it gives small average gains, but the improvements are often within the run-to-run noise.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Per-dataset gate hyperparameters (δ, h) are selected without a disclosed protocol; selection across δ∈{0.5,1,1.5,2} and h∈{16,32,64,128} could inflate Gated SRP's reported gains over comparators.","rationale":"The reader identifies the under-documented per-dataset hyperparameter selection as the weakest assumption, and I agree that this is the most load-bearing point. The paper's central contribution is an empirical improvement claim, and that claim is established only by comparing mean metrics across attention variants. If the per-dataset δ and h values were selected using the test split, or even using validation in a way that is not disclosed, the reported ranking and improvement counts lose their evidential force. The manuscript's own statement that values are 'fixed before final result aggregation' is not a protocol description; it does not say how the values were chosen or which data were used. This is not an accusation of misconduct: the selection may well have been done properly on validation, but the omission prevents a reader from verifying that. A second, reinforcing concern is the absence of paired significance testing; Table 13 shows that the seed-to-seed standard deviations are comparable to or larger than the mean gains, so the five-seed means alone are not strong evidence of a stable effect. The proposed concrete test addresses both by fixing hyperparameters and by checking paired differences. The verdict should remain conditional: the method is well-specified, cheap, and plausibly useful, but the headline empirical claim should be accepted only after the selection protocol and stability of the gains are established.","tokens_in":23686,"tokens_out":4327,"duration_ms":50571,"concrete_test":"Ask the authors to report the full hyperparameter selection protocol (candidate grid, split used for selection, and objective) and then rerun the main experiments with one fixed configuration, e.g., δ=1.0 and h=64, for all five TCGA cohorts and all five classification datasets under the same five seeds and the same validation-based checkpoint selection. If Gated SRP no longer achieves the highest mean C-index on all five cohorts, or if the mean improvement over NA collapses, the headline claim must be qualified as configuration-dependent. Additionally, report paired seed-level differences between Gated SRP and NA for each cohort and run a paired permutation test across the five seeds; if the gains are not significant on most cohorts, the claim of consistent improvement is not established.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim is that Gated SRP attains the highest mean C-index on all five TCGA survival cohorts and improves 12 of 16 classification metrics. The support for this claim depends on the per-dataset hyperparameters δ and h listed in Tables 10 and 11. Section B ('Hyperparameter Protocol') states only that 'the selected values are fixed before final result aggregation'; it does not describe the search grid, the split used for selection, or the selection criterion. With δ∈{0.5,1,1.5,2} and h∈{16,32,64,128}, there are 16 configurations per dataset; if test labels or the final test split influenced this selection, the reported mean gains and the across-cohort ranking could be overstated. Even under the benign reading that selection was done on validation, the comparison is not equally matched: standard NA, XSA, and Diff are assigned no per-dataset configuration search, while Gated SRP receives dataset-specific tuning that includes both the correction range and gate capacity. The reported improvements are also modest relative to the five-seed standard deviations in Table 13 (e.g., KIRP mean gain 0.040 vs. Gated SRP SD 0.081; LUAD gain 0.032 vs. SD 0.049), and no paired significance test is reported. Because the method's identity initialization guarantees that β=0 recovers the base attention, the only evidence that the learned correction helps is the mean-difference pattern; that pattern is directly vulnerable to undocumented configuration selection.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Gated Spatial Redundancy Projection (Gated SRP), a lightweight post-attention module for pathology transformers. For each patch token and attention head, the module computes a local redundancy axis from neighbouring value vectors, projects the attention output onto that axis, and subtracts a signed, token-specific learned coefficient times the projected component. The module is designed to reduce to standard attention at initialization through zero initialization of the gate, and it adds about 0.02% parameters. The authors evaluate Gated SRP against base Nyström attention, XSA, and differential attention on five TCGA survival cohorts and five slide-level classification datasets, reporting the highest mean C-index on all survival cohorts and improvements on 12 of 16 classification metrics. They also provide ablations on the gate range, gate gradients, patch encoder, gate factorization, gate initialization, and architecture choice, plus qualitative pathologist review of attention maps.","tokens_in":24022,"tokens_out":2787,"duration_ms":31318,"significance":"If the empirical claims are supported, Gated SRP is a useful, conceptually clean contribution to computational pathology: it identifies a pathology-specific failure mode (local spatial redundancy), proposes a geometrically interpretable correction, preserves the base attention path exactly at initialization, and adds negligible parameters. The identity initialization and the public code release are particular strengths, and the ablation suite is reasonably comprehensive. However, the headline empirical claim currently rests on mean differences across five seeds that are not accompanied by significance testing, and the per-dataset selection of two gate hyperparameters is not disclosed in a way that rules out test-set influence. These issues make the significance contingent on additional statistical and protocol evidence rather than established by the present manuscript.","major_comments":[{"comment":"The per-dataset selection of the gate range δ and gate hidden dimension h is not documented beyond the statement that 'the selected values are fixed before final result aggregation.' The paper does not report the search grid, the split used for selection, or the selection criterion. Since the text lists δ ∈ {0.5, 1, 1.5, 2} and h ∈ {16, 32, 64, 128}, this is a 16-configuration search per dataset or cohort; if the test split or test labels influenced this selection, the reported mean gains and the across-cohort ranking could be overstated. This concern is load-bearing because the central claim of 'highest mean C-index on all five cohorts' depends directly on these chosen configurations. The authors should disclose the full selection procedure, use a nested or non-overlapping validation split for configuration selection, and report results for a single fixed configuration as a sensitivity check.","section":"Section B, Hyperparameter Protocol; Tables 10 and 11"},{"comment":"No paired significance tests or confidence intervals are reported, and the magnitudes of the gains are small relative to the five-seed standard deviations. For example, in Table 13, the KIRP mean gain is 0.0401 while Gated SRP's standard deviation is 0.0805, and the LUAD gain is 0.0319 with a standard deviation of 0.0492; all five survival gains are smaller than one standard deviation. Given that the same seeds are used for all methods, paired comparisons are available and should be reported. Without such tests, the claim that Gated SRP 'obtains the highest mean C-index' is fragile, and the '12 of 16 metrics' claim in the abstract is not statistically supported.","section":"Appendix D, Tables 13 and 14; Tables 1 and 2"},{"comment":"The comparison is not equally matched: standard NA, XSA, and Diff do not receive any per-dataset configuration search, while Gated SRP receives dataset-specific tuning of both the correction range and gate capacity. Even under the benign reading that selection was performed on a validation split, the reported advantage could reflect the benefit of tuning rather than the mechanism. The authors should either run a comparable hyperparameter search for each baseline, or clearly report the selection protocol and demonstrate that the relative ordering is stable across a fixed configuration of δ and h for Gated SRP.","section":"Section 4.1 and Section B, Hyperparameter Protocol"}],"minor_comments":[{"comment":"The captions say green/red indicate 'meaningful' increase/decrease relative to the base NA, but no threshold or statistical definition of 'meaningful' is given; please define this term or remove it.","section":"Tables 1 and 2 captions"},{"comment":"The panel labels in Figure 3 appear garbled ('zre f l zpro j ziden =y zanti'), and Figure 2's panel layout is described in the caption but not clearly labeled in the text; please fix the rendering and refer to panels explicitly where needed.","section":"Figure 3 and Figure 2"},{"comment":"The text says n=3 gives an 8-neighbour grid for patch tokens, but the abstract and Figure 1 describe a '3×3-neighbour mean'; for patch-level images the 'analogous fixed grid' is not defined precisely. Please clarify how the neighbourhood is constructed when patch coordinates are unavailable.","section":"Section 3.3, Eq. (6)"},{"comment":"The deviations from the original TransMIL architecture (width, depth, block topology, Nyström landmark count, disabled value-residual depthwise convolution, regularization) are extensive; the main text should point readers to Appendix B at the first mention of 'TransMIL-style' so that the actual baseline is not misread as the official TransMIL.","section":"Appendix B, Model architecture"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a well-defined module and a clean identity-initialization property, but the headline claim currently leans on untested mean differences and undisclosed per-dataset hyperparameter selection. If the authors can provide a documented validation-based selection protocol and paired statistical tests, the paper would be acceptable; without those, the risk of over-claimed results is substantial. Please ensure the hyperparameter disclosure is enforced in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: Gated SRP is a well-specified, cheap attention correction that targets a real property of WSIs, and the paper is honestly written. I'd send it to review, but the empirical claim needs another pass because the per-dataset hyperparameter selection is undocumented and the gains are small relative to seed-to-seed noise.\n\nWhat's new: XSA projects attention outputs along the token's own value vector; Gated SRP projects along a local neighbourhood mean and lets a learned signed gate decide how much to remove, with identity initialization recovering base attention. That's a genuine, modest extension. The module is ~0.02% parameters, and the experiments are broad: five TCGA survival cohorts, five classification datasets, and several ablations that support the design choices (signed gate, stop-gradient, zero init). The authors also report standard deviations over five seeds and split by patient/case to avoid leakage. That is better than much of the literature in this area.\n\nWhere I'd push back: the main empirical claim rests on mean differences that are small—C-index gains mostly between 0.01 and 0.04—while the seed standard deviations are often larger (e.g., KIRP gain 0.040 vs SD 0.081; LUAD 0.032 vs 0.049). No paired significance test is reported. More importantly, Section B says only that selected δ and h values are 'fixed before final result aggregation'; it doesn't say how they were chosen. Given that δ and h were searched over 16 combinations per dataset, and the baselines get no analogous per-dataset tuning, the comparison is not equally matched. That is the load-bearing soft spot. If the authors disclose a validation-based selection protocol and still see these gains, I'd take the empirical claim much more seriously.\n\nThe method itself is not circular, the identity path is a nice safety property, and the ablations behave sensibly. I don't see a fundamental flaw, just an unsealed empirical case. The qualitative pathologist review is informal, but it's not the basis of the claim.\n\nBottom line: for someone working on pathology transformers or attention correction, this is worth reading and citing. It deserves peer review, but the reviewer should ask for the tuning protocol and some form of uncertainty quantification on the main comparisons.","headline":"A clean, lightweight attention correction with a plausible pathology-specific motivation, but the empirical case is undercut by an undisclosed per-dataset hyperparameter selection and small gains relative to seed noise.","tokens_in":24525,"tokens_out":2543,"would_cite":true,"duration_ms":26984,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Gated SRP claims that a post-attention projection along a local neighborhood redundancy axis, with a learned signed gate, improves pathology transformer predictions at negligible parameter cost.","keywords":["whole-slide pathology","transformer attention","local spatial redundancy","self-attention correction","survival analysis","slide-level classification","gated projection","C-index"],"falsifier":"A decisive check is to hold out the test labels of all ten datasets, select $\\delta$ and $h$ using only validation-set search (for example, a small grid over $\\delta \\in \\{0.5,1,1.5,2\\}$ and $h \\in \\{16,32,64,128\\}$), and then evaluate once. If Gated SRP no longer improves mean C-index on all five survival cohorts and more than half of classification metrics under this protocol, the central claim of consistent gains is not supported. A complementary mechanistic check: measure the cosine similarity between patch tokens and their neighborhood mean after correction; the method predicts that tokens with large positive $\\beta$ should become less similar to the local mean while retaining predictive risk signal.","tokens_in":23514,"feed_emoji":"🔬","tokens_out":14533,"duration_ms":129677,"temperature":0.7,"pith_summary":"Whole-slide pathology images are full of near-duplicate tissue patches, and the paper argues that this local redundancy is a specific failure mode for transformer attention: neighboring patches get mixed into each other and dilute the subtle evidence that matters for diagnosis and prognosis. To counteract this, the paper proposes Gated Spatial Redundancy Projection (Gated SRP), a small post-attention module that, for each patch and attention head, estimates a local redundancy direction from neighboring value vectors, projects the attention output onto that direction, and subtracts a learned signed fraction of the projected component. Because the gate is initialized at zero, standard attention is exactly recoverable during training, and the module adds roughly 0.02% of total parameters. In the paper's experiments, this correction yields the highest mean concordance index among compared attention variants on all five tested survival cohorts and improves the base attention on 12 of 16 slide-level classification metrics. If correct, a negligible-cost geometric correction to attention outputs, rather than a new model or loss, can make pathology transformers more sensitive to diagnostically relevant deviations.","feed_headline":"Small attention fix lifts survival ranking in all five cohorts","feed_subtitle":"It projects away local tissue redundancy, improving 12 of 16 classification metrics for just 0.02% extra parameters.","key_machinery":"The load-bearing object is the gated spatial redundancy projection $z_{i,h} = y_{i,h} - \\beta_{i,h}\\,\\mathrm{proj}_{\\hat{r}_{i,h}}(y_{i,h})$ applied after each self-attention layer to patch tokens only. The redundancy axis $\\hat{r}_{i,h}$ is the L2-normalized mean of the neighboring value vectors in a $3\\times3$ spatial window, computed with stop-gradient so it acts as a fixed geometric reference; the signed coefficient $\\beta_{i,h} = \\delta \\tanh(\\ell_{i,h})$ comes from a factored gate logit combining a shared token-level MLP (tissue homogeneity, neighbor count, log-confidence) with head-level alignment features (cosine similarity to the axis, absolute alignment, log-magnitude) and per-layer/head biases. The gate is initialized so $\\beta=0$, making the original attention update exactly reachable, which means the module can only learn to deviate from base attention when the training signal supports it. This single equation converts a suspected failure mode into a parameterized geometric choice: identity, projection, anti-projection, or reflection of the neighborhood-aligned component.","core_discovery":"The central claim is that the local spatial redundancy of whole-slide images—neighboring patches that share tissue type, stain, texture, and cellular composition—can repeatedly inject redundant content into patch tokens inside self-attention layers, making subtle diagnostic or prognostic deviations less distinguishable. Gated SRP addresses this by defining, per token and head, a local redundancy axis $\\hat{r}$ as the L2-normalized mean of the neighboring value vectors (computed with stop-gradient), projecting the attention output $y$ onto that axis, and writing the corrected output $z = y - \\beta_{\\mathrm{eff}} \\mathrm{proj}_{\\hat{r}}(y)$, where $\\beta_{\\mathrm{eff}} = \\delta \\tanh(\\ell)$ is a bounded signed gate produced by a factored token/head logit. The paper reports that, with this correction, mean case-level C-index improves over the base attention on all five cohorts (for example, from 0.7247 to 0.7648 on one kidney cohort and from 0.5513 to 0.5832 on one lung cohort), and classification performance improves on 12 of 16 reported metrics, while the module contributes an average of +0.02% parameters. The paper also claims that ablations support each design choice: the learned signed gate outperforms fixed projection strengths and a nonnegative sigmoid gate, detached geometric features outperform live gradients, and zero initialization outperforms random initialization, so the correction is learned gradually from an identity start.","pith_inferences":["An untested consequence is that the same redundancy-projection mechanism should transfer to other imagery with near-duplicate neighboring patches, such as satellite surveys, wide-field microscopy, or video frames; confidence in the pathology-specific story would grow if Gated SRP improves those tasks too.","The signed gate's negative branch lets the model amplify alignment with the neighborhood mean; a concrete test is whether tokens with strongly negative $\\beta$ coincide with tumor-stroma boundaries or micro-environmental gradients in survival cohorts, which would connect the geometric correction to histology.","The current redundancy axis is unsupervised geometry—the mean of neighboring value vectors; extending it to use tissue-type segmentation labels (for example, marking the $3\\times3$ window as homogeneous or mixed in annotated class) could sharpen the axis without changing the architecture.","The paper trains on frozen patch features and lists pretraining as open work; if correct, injecting this correction during self-supervised pretraining may improve the base representations themselves, but that is an extrapolation beyond the reported experiments."],"forward_implications":["Gated SRP improves mean case-level C-index over the base attention on every one of the five survival cohorts, with reported gains such as 0.7247 to 0.7648 on one kidney cohort and 0.5513 to 0.5832 on one lung cohort.","Across five classification datasets, it improves 12 of 16 reported metrics, reaches the best mean AUC on three datasets, and takes the best F1 and accuracy on two of them.","The identity initialization makes base attention exactly reachable at the first step, so the module can fall back to the original attention whenever correction is not useful, at an average cost of +0.02% of total parameters.","Ablations attribute the gain to the learned signed gate: fixed projection strengths of 1, 2, and -1 are less reliable, and a nonnegative sigmoid gate underperforms the signed tanh gate.","The correction also improves all reported metrics in the dense-attention vision transformer experiments on two image tasks, indicating the mechanism is not tied to the approximate-attention backbone."],"supporting_citations":[{"why":"Defines the vision-transformer patch-token and classification-token self-attention structure that Gated SRP modifies.","marker":"[13]"},{"why":"Supplies the standard multi-head attention formulation whose per-head output is the target of correction.","marker":"[39]"},{"why":"Provides the landmark-based approximate attention used as the base attention and main baseline in the whole-slide experiments.","marker":"[40]"},{"why":"Provides the slide-level transformer aggregation recipe on which the four-layer base aggregator is built.","marker":"[37]"},{"why":"Introduces the attention-output projection idea that Gated SRP generalizes from a token's own value vector to a spatial neighborhood axis.","marker":"[44]"},{"why":"Supplies the paired-attention subtraction baseline compared across all survival and classification tables.","marker":"[43]"},{"why":"Provides the frozen pathology patch encoder whose feature vectors are used for all main whole-slide evaluations.","marker":"[9]"},{"why":"Provides the five public survival cohorts used for the case-level concordance-index evaluation.","marker":"[33]"}],"fun_headline_variants":["Gated SRP: small attention fix, top survival in all five cohorts","0.02% extra params, better survival in 5 cohorts, 12/16 metrics","Project away redundant tissue: top survival in 5 cohorts","Zero-init gate on attention: best survival in all 5 cohorts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains rest on the assumption that the per-dataset choices of the gate range $\\delta$ and gate hidden size $h$ (listed in Tables 10 and 11) were fixed honestly before observing test performance; the paper does not describe the selection procedure, so if those hyperparameters were tuned against the test labels, the mean improvements could be inflated.","fun_headline_variants_meta":{"raw":{"variants":["Gated SRP: small attention fix, top survival in all five cohorts","0.02% extra params, better survival in 5 cohorts, 12/16 metrics","Project away redundant tissue: top survival in 5 cohorts","Zero-init gate on attention: best survival in all 5 cohorts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000602,"raw_usage":{"total_tokens":2874,"prompt_tokens":1070,"completion_tokens":1804,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":686,"completion_tokens_details":{"reasoning_tokens":1723}},"tokens_in":686,"tokens_out":1804,"duration_ms":12963,"temperature":1.0,"reasoning_tokens":1723,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:37:28.656563+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive check is to hold out the test labels of all ten datasets, select $\\delta$ and $h$ using only validation-set search (for example, a small grid over $\\delta \\in \\{0.5,1,1.5,2\\}$ and $h \\in \\{16,32,64,128\\}$), and then evaluate once. If Gated SRP no longer improves mean C-index on all five survival cohorts and more than half of classification metrics under this protocol, the central claim of consistent gains is not supported. A complementary mechanistic check: measure the cosine similarity between patch tokens and their neighborhood mean after correction; the method predicts that tokens with large positive $\\beta$ should become less similar to the local mean while retaining predictive risk signal.","supporting_citations":[{"cited_title":"An image is worth 16x16 words: Trans- formers for image recognition at scale.ICLR, 2021","cited_arxiv_id":null,"evidence_quote":"Defines the vision-transformer patch-token and classification-token self-attention structure that Gated SRP modifies."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the standard multi-head attention formulation whose per-head output is the target of correction."},{"cited_title":"Towards a general-purpose foundation model for computational pathology.Na- ture Medicine, 2024","cited_arxiv_id":null,"evidence_quote":"Provides the frozen pathology patch encoder whose feature vectors are used for all main whole-slide evaluations."},{"cited_title":"The Cancer Genome Atlas Program (TCGA).https: //www.cancer.gov/ccg/research/genome-sequencing/tcga","cited_arxiv_id":null,"evidence_quote":"Provides the five public survival cohorts used for the case-level concordance-index evaluation."}],"review_version":2}