REVIEW 5 major objections 5 minor 8 references
State Space Models for Extractive Summarization in Low Resource Scenarios
T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A Mamba-based encoder with Poincare-distance compression and LoRA tuning produces competitive extractive summaries in low-resource settings, beating Copycat, MeanSum, and LexRank on ROUGE-2 and ROUGE-L.
desk verdict Plausible architecture, but the paper's own numbers contradict its central claim and the evaluation is circular, so it needs major rework before it can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the interaction of three components. Mamba is a state space model that uses a selective scan to compress sequence information while keeping inference cost linear in sequence length; it supplies the sentence and review semantics. Poincare compression is the paper's dimensionality-reduction step: spectral clustering puts the concatenated Mamba embeddings into clusters, and each embedding is then represented by a vector of Poincare distances to the cluster centroids, which a batch-normalized linear layer uses to predict relevance. LoRA freezes the pretrained weights and adds low-rank trainable matrices, so most of the model stays fixed during low-resource training. These pieces are chained into a single sentence-relevance classifier, and the selected sentences are paraphrased by BART.
What would settle it
A reader could settle the claim by training MPoincareSum and Copycat on the same 136-review set and comparing them on a fresh, independently annotated 300-review test set across several random seeds; if the ROUGE-2 and ROUGE-L lead does not persist, the paper's central claim fails.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that a state space model does not have to be used as a black-box encoder for extractive summarization: after Mamba produces representations for a review and a candidate sentence, projecting the concatenated vector into Poincare space and replacing it with distances to spectral-cluster centroids yields a compact feature set that a linear classifier can learn from with very little data. The ablations support this reading: removing the Poincare compression lowers ROUGE, swapping Mamba for a transformer encoder gives roughly similar results, and LoRA tuning beats full fine-tuning. The paper therefore claims that Mamba plus Poincare compression plus LoRA is a competitive lightweight architecture for low-resource extractive summarization, with the qualification that its ROUGE-1 score trails FewSum.
Load-bearing premise
The evaluation depends on the assumption that sentence labels built from ROUGE scores and semantic similarity to the reference summary are a trustworthy measure of what a good extractive summary should select; because ROUGE is also the test metric, the reported gains partly rest on that assumption.
Editorial extensions
If this is right
- State space encoders can substitute for attention-based encoders in low-resource extractive summarization without losing ROUGE-2 or ROUGE-L quality.
- Poincare-distance compression of dense Mamba embeddings is a workable dimensionality-reduction step for sentence-relevance classification.
- LoRA-style parameter-efficient tuning beats full fine-tuning when training data are very limited.
- The matched ROUGE-1 gap to FewSum shows that fluency-oriented generation still has an edge on unigram overlap, so the claimed advantage is specific to ROUGE-2 and ROUGE-L.
- The full pipeline runs within a single-GPU setting with 30 GB of RAM, which is realistic for low-resource practitioners.
Reading between the lines
- A next check the paper does not perform is whether the ROUGE/semantic-similarity labeling agrees with human judgments of salience; without that, the reported advantage should be read as an in-metric result rather than proof of better summaries.
- The same architecture could be tested on longer-document low-resource corpora, where Mamba's linear-time inference would give a clearer practical edge than it does on 128-token reviews.
- Because the Mamba-versus-BERT ablation is reported without variance, re-running it with multiple seeds would show whether the near-parity is stable or an artifact of selecting the best checkpoint.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MPoincareSum, an extractive summarization method for low-resource Amazon review data. It encodes review-sentence pairs with the Mamba state space model, applies spectral clustering with a claimed Poincaré distance to produce features, trains a linear classifier with LoRA, and paraphrases selected sentences with BART. The experiments report ROUGE-1, ROUGE-2, and ROUGE-L scores on a dataset of 136 training and 73 test reviews and claim that MPoincareSum outperforms several existing approaches, including Copycat, MeanSum, LexRank, and FewSum.
Significance. If the central claim were established, the paper would offer a useful lightweight architecture combining state space models, spectral clustering, and parameter-efficient fine-tuning for low-resource extractive summarization. The combination of components is plausible and the topic is relevant. However, the paper does not provide code or data, and the current evaluation has load-bearing flaws: the abstract's outperformance claim is contradicted by the paper's own Table 3, the training labels are derived from the same ROUGE metric used for evaluation, and model selection is performed on the test set. As submitted, the evidence is not sufficient to support the stated claims.
major comments (5)
- [Abstract and §4.3/Table 3] The abstract's central claim that MPoincareSum 'outperforms several existing approaches' is not supported by Table 3. MPoincareSum has the lowest ROUGE-1 score (0.214 versus 0.269–0.372 among the listed baselines) and is below FewSum on ROUGE-L (0.200 versus 0.227). The only consistent statement supported by the table is that MPoincareSum beats Copycat, MeanSum, and LexRank on ROUGE-2 and ROUGE-L, with margins of 0.012–0.021 and 0.017–0.040, respectively. The authors should restate the claim to match the table and assess whether such small margins are meaningful on a 73-review test set.
- [§4.1] The training labels are generated using ROUGE scores and semantic similarity between each sentence and its corresponding review, and the model is then evaluated with ROUGE-1, ROUGE-2, and ROUGE-L against the same reference summaries. Section 4.1 does not specify the threshold, procedure, or validation of these labels. This creates a circularity: the reported ROUGE gains partly measure how well the model recovers the metric used to construct its own training signal. The labeling pipeline should be independently validated, or the evaluation should use a metric not derived from the labeling procedure.
- [§4.2] The sentence 'Each experiment was repeated multiple times, and we selected the best-performing model' indicates that model selection was performed on the test set. With only 73 test reviews, best-run selection on the test set inflates ROUGE scores and makes the reported point estimates incomparable to the literature baselines, for which no such selection is reported. The authors need a fixed evaluation protocol: select hyperparameters and checkpoints on a validation split, report the mean and standard deviation over runs, and provide significance tests or confidence intervals.
- [Eq. (8)] Equation (8) states d_Poincare(a,b) = 1 + 2|a−b|² / ((1−|a|²)(1−|b|²)), which is not the standard Poincaré metric. The standard formula is arccosh(1 + 2|a−b|² / ((1−|a|²)(1−|b|²))). As written, the quantity is not a distance, and the paper does not verify that the Mamba embeddings lie in the unit ball. The purported benefit of 'Poincaré compression' is therefore not established by the reported experiments.
- [§7 Ablation Study] The ablation section repeatedly refers to 'Table 4', but no Table 4 appears in the manuscript. The claims that removing Poincaré compression, replacing Mamba with BERT, and switching from LoRA to full fine-tuning cause 'significant' ROUGE drops cannot be verified without the table, its exact numbers, and significance tests. This is a load-bearing omission because the ablation is used to justify the contribution of each proposed component.
minor comments (5)
- [Throughout] The section numbering is inconsistent: the Introduction is unnumbered, Related Work is numbered 5, Method is numbered 6, but the subsections are numbered 3.1–3.5. The numbering should be harmonized.
- [Contributions] The contributions list contains a typo: 'HiPPPO initialization' should read 'HiPPO initialization.'
- [References] The reference list mixes citation styles (e.g., 'Gu, A., & Dao, T. (2023)' versus 'Arthur Bražinskas et al., 2020') and contains two entries for Bražinskas et al. 2020 under slightly different spellings. A consistent style should be used.
- [Eq. (3)] The symbol ⊕ in Eq. (3) is not defined; clarify whether the paraphrases of individual sentences are concatenated and how the final summary length is controlled.
- [Table 1] The example in Table 1 is difficult to interpret because the intermediate extractive selection is not shown; it is unclear which sentences from the review were chosen and how BART paraphrased them into the displayed output.
Circularity Check
ROUGE-derived training labels are evaluated with the same ROUGE metric, partially circular; no load-bearing self-citations.
-
fitted input called prediction
[Section 4.1 (Amazon Review Dataset), Section 4.2 (Experiments Settings), Abstract]
"The annotation process was based on ROUGE scores and the semantic similarity between each sentence and its corresponding review. ... We evaluated our model using ROUGE -1, ROUGE -2, and ROUGE-L scores."
The binary relevance labels used to train the classifier are generated from ROUGE scores (plus semantic similarity), and the model's output summaries are scored with the same ROUGE metric. The concept 'relevant sentence' is thus defined by the evaluation metric: the model is trained to approximate a ROUGE-based labeling function, and then the ROUGE score of the selected sentences is reported as evidence of quality. This is not an independent measurement: a model trained on ROUGE-derived labels is, by construction, optimized toward the metric used to evaluate it, unlike the unsupervised baselines (Copycat, MeanSum, LexRank) that never see ROUGE-based supervision. The labeling threshold/procedure is omitted, so the ROUGE component of the labels cannot be separated from the evaluation.
full rationale
The paper's architectural derivation chain is built from externally cited components (Mamba, LoRA, spectral clustering, BART) and contains no load-bearing self-citations or imported uniqueness theorems; the architecture itself is not circular. The central circularity is in the evaluation protocol: Section 4.1 states that sentence relevance labels were 'based on ROUGE scores and the semantic similarity between each sentence and its corresponding review,' while the abstract and Section 4.2 state that performance was 'assessed using ROUGE scores' (ROUGE-1/2/L). Because the training target is itself a function of ROUGE, the reported ROUGE scores do not independently validate summarization quality, and the comparison against unsupervised baselines that did not receive ROUGE-derived supervision is unbalanced. The exact labeling threshold is not reported, so the ROUGE component cannot be disentangled from the labels. Additional concerns—best-run selection among repeated experiments (Section 4.2), the abstract's 'outperforms' claim contradicted by Table 3 (MPoincareSum has the lowest R1 at 0.214 versus 0.269–0.372 for baselines), and the missing arcosh in the Poincaré distance formula in Eq. 8—are correctness risks rather than circularity. Overall, one partial circular step in the training/evaluation loop; no self-citation chain forcing the result.
Assumptions & free parameters
free parameters (8)
- Learning rate =
2e-5
- Weight decay =
0.5
- Dropout rate =
0.5
- LoRA alpha =
32
- LoRA dropout =
0.1
- Maximum sequence length =
128
- Number of spectral clusters k =
Not reported
- Best-run selection on test set =
Selected among multiple runs
assumptions (6)
- domain assumption Sentence relevance can be reliably annotated using ROUGE scores and semantic similarity to the reference summary.
- ad hoc to paper The Poincare distance formula in Eq. (8) is the correct distance between embeddings.
- domain assumption A pre-trained Mamba-130m model provides better or comparable semantics than transformers for low-resource reviews.
- domain assumption The baseline results from FewSum, Copycat, MeanSum, and LexRank are directly comparable to the authors' results.
- domain assumption BART paraphrasing preserves the relevance and content of extracted sentences.
- standard math Spectral clustering with k-means on the graph Laplacian produces useful centroids for the embeddings.
Cite this review
Pith. "Pith review of State Space Models for Extractive Summarization in Low Resource Scenarios." pith.science (2026). https://pith.science/paper/BV42UO4B
@misc{pith2026250114673,
author = {Pith},
title = {Pith review of: State Space Models for Extractive Summarization in Low Resource Scenarios},
year = {2026},
howpublished = {\url{https://pith.science/paper/BV42UO4B}},
note = {Machine review of arXiv:2501.14673}
}
read the original abstract
Extractive summarization involves selecting the most relevant sentences from a text. Recently, researchers have focused on advancing methods to improve state-of-the-art results in low-resource settings. Motivated by these advancements, we propose the MPoincareSum method. This method applies the Mamba state space model to generate the semantics of reviews and sentences, which are then concatenated. A Poincare compression is used to select the most meaningful features, followed by the application of a linear layer to predict sentence relevance based on the corresponding review. Finally, we paraphrase the relevant sentences to create the final summary. To evaluate the effectiveness of MPoincareSum, we conducted extensive experiments using the Amazon review dataset. The performance of the method was assessed using ROUGE scores. The experimental results demonstrate that MPoincareSum outperforms several existing approaches in the literature
Figures
Reference graph
Works this paper leans on
-
[3]
Meta-Transfer Learning for Low-Resource Abstractive Summarization
Few-Shot Learning for Opinion Summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4119–4135, Online. Association for Computational Linguistics.https://doi.org/10.18653/v1/2020.emnlp -main.337 7 Chen, Y., & Shuai, H. (2021). Meta-Transfer Learning for Low -Resource Abstractive Summarizatio...
work page Pith review arXiv 2021
-
[4]
On Optimal Transformer Depth for Low-Resource Language Translation
On optimal transformer depth for low - resource language translation, Computing Research Repository, ArXiv, , abs/2004.04418. Eric Chu and Peter Liu. 2019 . Meansum: a neural model for unsupervised multi-document abstractive summarization. In Proceedings of International Conference on Machine Learning (ICML), pages 1223–1232. https://doi.org/10.48550/arXi...
work page Pith review arXiv 2023
-
[5]
BART: Denoising Sequence -to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871 –7880, Online. Association for Computational Linguistics.https://doi.org/10.18653/v1/2020.acl- main.703 Nianlong Gu, Elliott Ash, and Rich...
-
[7]
Stepwise Extractive Summarization and Planning with Structured Transformers . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4143 –4159, Online . Association for Computational Linguistics.https://doi.org/10.18653/v1/2020.emnl p-main.339 Smith, L.N. (2015). Cyclical Learning Rates for Training Neura...
-
[8]
AdaptSum: Towards Low -Resource Domain Adaptation for Abstractive Summarization . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5892 –5904, Online. Association for Computational Linguistics. Yongtai Liu, Joshua Maynez, Gonçalo Simões, and Shashi Nara...
work page 2021
-
[2019]
Investigating Meta -Learning Algorithms for Low-Resource Natural Language Understanding Tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1192–1197, Hong Kong, China. Association for Computational Linguistics . htt...
-
[2021]
Long Document Summarization in a Low Resource Setting using Pretrained Language Models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop, pages 71 –80, Online. Association for Computational Linguistics. Alan Akbik...
work page 2018
-
[2022]
MemSum: Extractive Summarization of Long Documents Using Multi -Step Episodic Markov Decision Processes . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6507 –6522, Dublin, Ireland. Association for Computational Linguistics.https://doi.org/10.18653/v1/2022.acl- long.450 Nickel, M.,...
arXiv 2017
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.