REVIEW 4 major objections 6 minor 30 references
eccDNAMamba: A Pre-Trained Model for Ultra-Long eccDNA Sequence Analysis
T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read eccDNAMamba, a bidirectional state-space encoder for circular DNA, classifies cancer- and healthy-origin eccDNA and separates authentic eccDNA from pseudo-circular fragments while processing sequences up to 200,000 base pairs.
desk verdict A reasonable model with an unverified 200-kbp scaling claim and a circular-augmentation ablation that doesn't test the actual design; worth review but not the headline as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a bidirectionally fused Mamba-2 state-space encoder, a linear-time sequence model that updates a hidden state through input-dependent gating rather than quadratic attention. Two copies of the encoder read the same tokenized sequence left-to-right and right-to-left, and a shared MLP merges the aligned hidden states so every token has both upstream and downstream context. Three supporting mechanisms carry the design: byte-pair encoding compresses DNA into motif-like tokens at roughly 5.16 bases per token, circular augmentation appends the first 64 tokens to the sequence end to model head-tail continuity, and a SpanBERT-style span-masking objective trains the model to reconstruct contiguous spans rather than isolated tokens. Padding is suppressed by zeroed embeddings, attention masks, and reset hidden states at padding positions.
What would settle it
Re-run the reported fine-tuning protocol on the 10–200 kbp cancer-versus-healthy and authentic-versus-pseudo tasks with the 64-token circular augmentation removed or replaced by the same number of zero or random padding tokens, holding all other settings fixed. If accuracy, macro-F1, and recall remain within noise, the circular-specific component contributes nothing and the model reduces to a bidirectional BPE-Mamba; a large drop would confirm that head-tail token dependencies are learnable and load-bearing.
Extended reading notes
Core claim
eccDNAMamba is, by the authors' account, the first bidirectional state-space encoder tailored for circular eccDNA. It tokenizes DNA with byte-pair encoding (about 5.16 bases per token), appends the first 64 tokens of a sequence to its end to preserve head-tail circular dependencies, and feeds the sequence through forward and reverse Mamba-2 encoders whose hidden states are aligned and merged by a shared MLP. Pre-training uses a SpanBERT-style span-masking objective on roughly 100 million tokens from human and cross-species eccDNA corpora. Fine-tuned on two classification benchmarks, the model maintains accuracy on sequences up to 200 kilobase pairs while baseline models that truncate inputs or operate at single-nucleotide resolution degrade. A motif analysis of the cancer classifier suggests its decisions turn on CG-rich, C2H2 zinc-finger-like motifs, with AT-rich false negatives pointing to an alternative regulatory logic.
Load-bearing premise
The load-bearing premise is that appending the first 64 tokens of an eccDNA sequence to its end preserves biologically meaningful head-tail dependencies at the 10–200 kilobase-pair scale; the only ablation offered for this uses full-length self-concatenation on toy models with sequences under 200 base pairs, not the 64-token prefix scheme on the actual pretrained model.
Editorial extensions
If this is right
- If the central claim holds, full-length circular eccDNA molecules up to 200 kilobase pairs can be encoded end-to-end at linear time and memory cost, without truncation or arbitrary breakpoints.
- A BPE-tokenized bidirectional state-space encoder can beat nucleotide-resolution and transformer-based genomic foundation models on eccDNA classification despite being pre-trained on far fewer tokens.
- The cancer-versus-healthy result implies that cancer-derived and healthy eccDNAs carry learnable sequence-level differences, concentrated in CG-rich zinc-finger-like motifs, rather than being indistinguishable circular fragments.
- The authentic-versus-pseudo result implies that real eccDNAs contain non-random sequence features beyond length distribution, since length-matched random circularized genomic fragments are separable at about 0.74 macro-F1.
- The model's stable performance on 10-200 kbp inputs where baselines degrade indicates that the practical scaling path for long circular genomes is linear-time state-space encoding paired with motif-level tokenization, not longer attention contexts.
Reading between the lines
- If the 64-token circular augmentation is what sustains long-sequence accuracy, a rotationally invariant or periodic positional encoding should match or exceed its effect; comparing those alternatives would separate the circular-wrapping trick from the value of bidirectional context alone.
- At roughly 5.16 base pairs per token, a 200-kbp molecule becomes about 38,000 tokens, so a nucleotide-resolution model with the same token budget would need over five times more context; this suggests BPE compression may contribute as much as the architecture to the reported scaling results.
- The motif analysis suggests a testable biological extension: cancer eccDNAs enriched for ZNF24 and ZNF263 binding motifs should show higher transcriptional activity or amplification in matched tumor data, checkable with expression or chromatin assays.
- Binning the authentic-versus-pseudo task by sequence length (10–50, 50–100, and 100–200 kbp) would reveal whether the model's advantage grows with full-length context, isolating the value of avoiding truncation.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes eccDNAMamba, a bidirectional Mamba-2 encoder pretrained on a corpus of ~120,000 eccDNA sequences with a span-masked language modeling objective, BPE tokenization, and a circular augmentation strategy that appends the first 64 tokens to the end of each sequence. The model is evaluated on two binary classification tasks: distinguishing cancer-derived from healthy eccDNA, and distinguishing authentic eccDNA from length-matched pseudo-circular genomic fragments. The authors report state-of-the-art accuracy, F1, precision, and recall against DNABERT-2, HyenaDNA, Caduceus, and DeepCircle, and claim that the model processes full-length sequences up to 200 kbp without performance degradation. A supplementary motif analysis links the model's cancer predictions to CG-rich zinc-finger transcription factor binding motifs.
Significance. If the empirical claims hold, eccDNAMamba would be a genuinely useful resource: a linear-time bidirectional encoder that can ingest full-length circular eccDNA molecules, trained on a domain-specific corpus and released with code. The paper is transparent about many pretraining details, uses two real biological datasets, and the motif analysis provides a falsifiable biological hypothesis. The main contributions are the bidirectional Mamba-2 encoder adapted for circular DNA and the demonstration of strong classification performance. However, the significance is conditional on resolving three load-bearing concerns: the 200-kbp full-length claim is not supported by the reported token budget; the baseline comparisons are confounded by in-domain pretraining; and the circular augmentation is not validated on the actual model at long sequence lengths.
major comments (4)
- [§4.5 and Abstract] The claim that eccDNAMamba 'scales to sequences up to 200 Kbp' and 'processes complete sequences end-to-end' is not supported by the reported setup. In §4.5, the comparison fixes eccDNAMamba at 30,000 BPE tokens. At the reported average compression of 5.16 bp/token (§4.1, §A.3), 30,000 tokens correspond to roughly 154,800 bp, so sequences in the upper part of the 10–200 kbp range cannot be fed in full. The manuscript does not report the token counts of the long test sequences or the model's max_length. Please report the actual length distribution and token counts of the test set, or revise the scaling and 'full-length end-to-end' claims accordingly.
- [§4.2, §4.5, §4.6] The central comparison is confounded by in-domain pretraining. eccDNAMamba is pretrained on ~101.5 million tokens of eccDNA, while DNABERT-2, HyenaDNA, Caduceus, and DeepCircle are used as released, without continued pretraining on the eccDNA corpus. The observed performance gap may therefore reflect domain adaptation rather than the proposed architecture, circular augmentation, or tokenization. The claim that eccDNAMamba 'consistently outperforms' these models needs a control: for example, continue pretraining the baselines on the same eccDNA corpus, or evaluate eccDNAMamba without eccDNA-specific pretraining, and compare under matched conditions.
- [§3.2.3 and Appendix A.1] The circular augmentation strategy is load-bearing for the 'tailored for circular DNA' claim, but the supporting ablation in Appendix A.1 does not test the actual design. The ablation uses CNN, MLP, and Mamba models on sequences under 200 bp with full-length self-concatenation, rather than the 64-token prefix augmentation applied to the pretrained eccDNAMamba at 10–200 kbp. The differences in Table 3 (e.g., Mamba with versus without augmentation: 0.8048 vs 0.8011 F1) also appear small relative to the reported standard deviations. Please provide an ablation on the real model and long sequences, or explicitly limit the claim that circular augmentation is what drives the long-sequence results.
- [Table 2, §4.5] The main results table reports single runs without error bars or significance tests. Several comparisons are close: on the <10 kb cancer task, eccDNAMamba's macro F1 is 0.8242 versus Caduceus's 0.8216, and on precision Caduceus is higher (0.8248 vs 0.8242). The strong conclusion that eccDNAMamba is 'the only model that remains robust across all evaluation metrics' needs multiple seeds with standard deviations, or at minimum a statistical test of the differences, before it can be considered established.
minor comments (6)
- [Table 2 caption] The caption says 'Performance comparison of eccDNAMamba and DNABERT-2 under different training settings,' but the table includes HyenaDNA, Caduceus, and DeepCircle; please correct the caption to reflect all compared models.
- [§4.6 and Table 2] The DeepCircle zero-shot row is listed with a training set of 20,000 sequences, which is inconsistent with the notion of zero-shot evaluation. Please clarify whether zero-shot means no fine-tuning on this task and, if so, remove the training-set entry or explain how the training set was used.
- [§2.2] The citation 'Li et al. (Chang et al., 2023)' is confusingly formatted; it appears to refer to the DeepCircle paper by Chang et al., 2023, but the in-text name and the reference entry should be aligned.
- [§5 and Figure 3] The numbers '28, 100 TP sequences using 29, 292 TN as background' are ambiguous due to spacing; use commas consistently (e.g., 28,100 and 29,292) to avoid misreading.
- [Appendix A.2] The appendix says the augmentation appends 'approximately 25% of the sequence,' while the main text (§3.2.3) specifies a fixed 64 tokens. Please clarify whether the prefix length is fixed at 64 tokens or scales with sequence length, and reconcile the two descriptions.
- [Throughout] There are several typos: 'HeynaDNA' should be 'HyenaDNA', 'eccDNAMamaba' should be 'eccDNAMamba', and 'state of art(SOTA)' should be 'state-of-the-art (SOTA)'.
Circularity Check
No significant circularity: the model's predictions are evaluated on external held-out labels, and its design choices (BPE, span masking, bidirectional Mamba, circular augmentation) are defined independently of the downstream classification targets.
full rationale
eccDNAMamba is an empirical deep-learning paper with no derivation chain that reduces to its own inputs. The central components are defined independently of the target labels: BPE tokenization is fit on unlabeled eccDNA corpora, the bidirectional Mamba encoder is initialized from mamba2-130m and trained from scratch with a SpanMLM objective, and the circular augmentation appends the first 64 tokens to the end of each sequence. None of these steps is defined in terms of the cancer-vs-healthy or authentic-vs-pseudo labels, so no self-definitional reduction occurs. The fine-tuning evaluation compares predictions against ground-truth labels on independent test sets (Sections 4.5 and 4.6), so the reported accuracies, F1 scores, precisions, and recalls are not fitted inputs renamed as predictions. The paper contains no load-bearing self-citations: all cited methods (Mamba, DNABERT-2, HyenaDNA, Caduceus, SpanBERT, DeepCircle, CircleBase, eccDNA Atlas) are external works, and the authors' own prior contributions are not invoked to justify the architecture or forbid alternatives. The only self-referential empirical content is the circular-augmentation ablation in Appendix A.1, which explicitly acknowledges that it uses toy CNN/MLP/Mamba models on sequences under 200 bp as a practical substitute for repeated pretraining; this is a stated limitation of the design validation, not a circular derivation of the downstream results. The in-domain pretraining overlap with CircleBase and eccDNA Atlas could create a benchmark-confounding concern, but that is a data-ecology issue rather than a circular-reasoning issue under the rules. Concerns about the 200 kbp scaling claim arising from the 30,000-BPE-token comparison in Section 4.5 concern verification of the setup, not equivalence of the result to its inputs. Overall, the paper's empirical claims are self-contained against external benchmarks in the sense relevant to circularity analysis.
Assumptions & free parameters
free parameters (6)
- Circular augmentation prefix length s =
64 tokens
- Span length and mask ratio =
Average span 3 tokens; 15% tokens masked
- Pretraining learning rate, batch size, epochs =
5e-4, effective batch 144, 3 epochs
- BPE vocabulary size =
Not specified
- Pretraining length filter =
10 kbp
- Mamba-2 hidden configuration =
state-spaces/mamba2-130m
assumptions (5)
- domain assumption eccDNA head-tail dependencies are biologically meaningful and preserved by appending a 64-token prefix.
- domain assumption BPE tokenization preserves motif-level information sufficient for classification.
- standard math Mamba-2 state-space layers capture long-range dependencies on DNA as claimed.
- domain assumption Span masking pretraining transfers to downstream eccDNA classification.
- domain assumption Pretraining on sequences under 10 kbp transfers to full-length sequences up to 200 kbp after fine-tuning.
Cite this review
Pith. "Pith review of eccDNAMamba: A Pre-Trained Model for Ultra-Long eccDNA Sequence Analysis." pith.science (2026). https://pith.science/paper/JESYLAC6
@misc{pith2026250618940,
author = {Pith},
title = {Pith review of: eccDNAMamba: A Pre-Trained Model for Ultra-Long eccDNA Sequence Analysis},
year = {2026},
howpublished = {\url{https://pith.science/paper/JESYLAC6}},
note = {Machine review of arXiv:2506.18940}
}
read the original abstract
Extrachromosomal circular DNA (eccDNA) plays key regulatory roles and contributes to oncogene overexpression in cancer through high-copy amplification and long-range interactions. Despite advances in modeling, no pre-trained models currently support full-length circular eccDNA for downstream analysis. Existing genomic models are either limited to single-nucleotide resolution or hindered by the inefficiency of the quadratic attention mechanism. Here, we introduce eccDNAMamba, the first bidirectional state-space encoder tailored for circular DNA sequences. It combines forward and reverse passes for full-context representation learning with linear-time complexity, and preserves circular structure through a novel augmentation strategy. Tested on two real-world datasets, eccDNAMamba achieves strong classification performance and scales to sequences up to 200 Kbp, offering a robust and efficient framework for modeling circular genomes. Our codes are available at https://github.com/zzq1zh/GenAI-Lab.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Bailey, T. L. Streme: accurate and versatile sequence motif discovery. Bioinformatics, 37 0 (18): 0 2834--2840, 2021
work page 2021
-
[2]
Extrachromosomal circular dna: category, biogenesis, recognition, and functions
Cao, X., Wang, S., Ge, L., Zhang, W., Huang, J., and Sun, W. Extrachromosomal circular dna: category, biogenesis, recognition, and functions. Frontiers in veterinary science, 8: 0 693641, 2021
work page 2021
-
[3]
Chang, K.-L., Chen, J.-H., Lin, T.-C., Leu, J.-Y., Kao, C.-F., Wong, J. Y., and Tsai, H.-K. Short human eccdnas are predictable from sequences. Briefings in Bioinformatics, 24 0 (3): 0 bbad147, 04 2023. ISSN 1477-4054. doi:10.1093/bib/bbad147. URL https://doi.org/10.1093/bib/bbad147
- [4]
-
[5]
Extrachromosomal dna (ecdna) in cancer: mechanisms, functions, and clinical implications
Dong, Y., He, Q., Chen, X., Yang, F., He, L., and Zheng, Y. Extrachromosomal dna (ecdna) in cancer: mechanisms, functions, and clinical implications. Frontiers in oncology, 13: 0 1194405, 2023
work page 2023
-
[6]
Fang, M., Fang, J., Luo, S., Liu, K., Yu, Q., Yang, J., Zhou, Y., Li, Z., Sun, R., Guo, C., and Qu, K. eccdna-pipe: an integrated pipeline for identification, analysis and visualization of extrachromosomal circular dna from high-throughput sequencing data. Briefings in Bioinformatics, 25 0 (2): 0 bbae034, 02 2024. ISSN 1477-4054. doi:10.1093/bib/bbae034. ...
-
[7]
A new algorithm for data compression
Gage, P. A new algorithm for data compression. The C Users Journal, 12 0 (2): 0 23--38, 1994
1994
-
[8]
Gu, A. and Dao, T. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023
arXiv 2023
Show all 30 references
-
[9]
A., Bailey, T
Gupta, S., Stamatoyannopoulos, J. A., Bailey, T. L., and Noble, W. S. Quantifying similarity between motifs. Genome biology, 8: 0 1--9, 2007
2007
-
[10]
Bidirectional lstm-crf models for sequence tagging
Huang, Z., Xu, W., and Yu, K. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991, 2015
2015 arXiv
-
[11]
S., Zettlemoyer, L., and Levy, O
Joshi, M., Chen, D., Liu, Y., Weld, D. S., Zettlemoyer, L., and Levy, O. Spanbert: Improving pre-training by representing and predicting spans. Transactions of the association for computational linguistics, 8: 0 64--77, 2020
2020
-
[12]
P., Rodriguez-Fos, E., Helmsauer, K., Burkert, M., MacArthur, I
Koche, R. P., Rodriguez-Fos, E., Helmsauer, K., Burkert, M., MacArthur, I. C., Maag, J., Chamorro, R., Munoz-Perez, N., Puiggr \`o s, M., Dorado Garcia, H., et al. Extrachromosomal circular dna drives oncogenic genome remodeling in neuroblastoma. Nature genetics, 52 0 (1): 0 2...
2020
-
[13]
Transcription factor znf263 enhances egfr-targeted therapeutic response and reduces residual disease in lung adenocarcinoma
Liang, J., Bi, G., Sui, Q., Zhao, G., Zhang, H., Bian, Y., Chen, Z., Huang, Y., Xi, J., Shi, Y., et al. Transcription factor znf263 enhances egfr-targeted therapeutic response and reduces residual disease in lung adenocarcinoma. Cell Reports, 43 0 (2), 2024
2024
-
[14]
Small extrachromosomal circular dna (eccdna): major functions in evolution and cancer
Ling, X., Han, Y., Meng, J., Zhong, B., Chen, J., Zhang, H., Qin, J., Pang, J., and Liu, L. Small extrachromosomal circular dna (eccdna): major functions in evolution and cancer. Molecular Cancer, 20 0 (1): 0 113, 2021. doi:10.1186/s12943-021-01413-8. URL https://doi.org/10.11...
2021 doi
-
[15]
L., Takata, M., Gon c alves-Carneiro, D., Keane, S
Meagher, J. L., Takata, M., Gon c alves-Carneiro, D., Keane, S. C., Rebendenne, A., Ong, H., Orr, V. K., MacDonald, M. R., Stuckey, J. A., Bieniasz, P. D., et al. Structure of the zinc-finger antiviral protein in complex with rna reveals a mechanism for selective targeting of ...
2019
-
[16]
P., Richard, G., Caton, J., Lopez Carranza, N., Skwark, M., et al
Mendoza-Revilla, J., Trop, E., Gonzalez, L., Roller, M., Dalla-Torre, H., de Almeida, B. P., Richard, G., Caton, J., Lopez Carranza, N., Skwark, M., et al. A foundational large language model for edible plant genomes. Communications Biology, 7 0 (1): 0 835, 2024
2024
-
[17]
A., and Ré, C
Nguyen, E., Poli, M., Faizi, M., Thomas, A., Birch-Sykes, C., Wornow, M., Patel, A., Rabideau, C., Massaroli, S., Bengio, Y., Ermon, S., Baccus, S. A., and Ré, C. Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution, 2023. URL https://arxiv.org/abs/2306.15794
2023 arXiv
-
[18]
Retracted: A novel tumor suppressor gene, znf24, inhibits the development of nsclc by inhibiting the wnt signaling pathway to induce cell senescence
Pang, B., Wang, Y., and Chang, X. Retracted: A novel tumor suppressor gene, znf24, inhibits the development of nsclc by inhibiting the wnt signaling pathway to induce cell senescence. Frontiers in Oncology, 11: 0 664369, 2021
2021
-
[19]
New insights of extrachromosomal dna in tumorigenesis and therapeutic resistance of cancer
Qiu, H., Shao, Z.-Y., Wen, X., and Zhang, L.-Z. New insights of extrachromosomal dna in tumorigenesis and therapeutic resistance of cancer. American Journal of Cancer Research, 10 0 (12): 0 4056, 2020
2020
-
[20]
H., Gokaslan, A., Dao, T., Gu, A., and Kuleshov, V
Schiff, Y., Kao, C. H., Gokaslan, A., Dao, T., Gu, A., and Kuleshov, V. Caduceus: Bi-directional equivariant long-range dna sequence modeling. In International Conference on Machine Learning, pp.\ 43632--43648. PMLR, 2024
2024
-
[21]
Neural machine translation of rare words with subword units
Sennrich, R., Haddow, B., and Birch, A. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909, 2015
2015 arXiv
-
[22]
K., Mukherjee, S., Ebert, B
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proceedings o...
2005
-
[23]
M., Deshpande, V., Beyter, D., Koga, T., Rusert, J., Lee, C., Li, B., Arden, K., Ren, B., Nathanson, D
Turner, K. M., Deshpande, V., Beyter, D., Koga, T., Rusert, J., Lee, C., Li, B., Arden, K., Ren, B., Nathanson, D. A., et al. Extrachromosomal oncogene amplification drives tumour evolution and genetic heterogeneity. Nature, 543 0 (7643): 0 122--125, 2017
2017
-
[24]
T., Yang, A., Albu, M., Cote, A
Weirauch, M. T., Yang, A., Albu, M., Cote, A. G., Montenegro-Montero, A., Drewe, P., Najafabadi, H. S., Lambert, S. A., Mann, I., Cook, K., Zheng, H., Goity, A., van Bakel, H., Lozano, J. C., Galli, M., Lewsey, M. G., Huang, E., Mukherjee, T., Chen, X., Reece-Hoyes, J. S., Gov...
2014
-
[25]
M., Nguyen, N., Raviram, R., Erb, M., Santini, J., Luebeck, J., Rajkumar, U., Diao, Y., Li, B., et al
Wu, S., Turner, K. M., Nguyen, N., Raviram, R., Erb, M., Santini, J., Luebeck, J., Rajkumar, U., Diao, Y., Li, B., et al. Circular ecdna promotes accessible chromatin and high oncogene expression. Nature, 575 0 (7784): 0 699--703, 2019
2019
-
[26]
Circlebase: an integrated resource and analysis platform for human eccdnas
Zhao, X., Shi, L., Ruan, S., Bi, W., Chen, Y., Chen, L., Liu, Y., Li, M., Qiao, J., and Mao, F. Circlebase: an integrated resource and analysis platform for human eccdnas. Nucleic acids research, 50 0 (D1): 0 D72--D82, 2022 a
2022
-
[27]
Extrachromosomal circular dna: Current status and future prospects
Zhao, Y., Yu, L., Zhang, S., Su, X., and Zhou, X. Extrachromosomal circular dna: Current status and future prospects. Elife, 11: 0 e81412, 2022 b
2022
-
[28]
eccdna atlas: a comprehensive resource of eccdna catalog
Zhong, T., Wang, W., Liu, H., Zeng, M., Zhao, X., and Guo, Z. eccdna atlas: a comprehensive resource of eccdna catalog. Briefings in Bioinformatics, 24 0 (2): 0 bbad037, 2023
2023
-
[29]
Dnabert-2: Efficient foundation model and benchmark for multi-species genome
Zhou, Z., Ji, Y., Li, W., Dutta, P., Davuluri, R., and Liu, H. Dnabert-2: Efficient foundation model and benchmark for multi-species genome. arXiv preprint arXiv:2306.15006, 2023
2023 arXiv
-
[30]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.