REVIEW 4 major objections 5 minor 42 references
FaRMamba: Frequency-based learning and Reconstruction aided Mamba for Medical Segmentation
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read FaRMamba claims to fix Vision Mamba's loss of high-frequency detail and 2D spatial structure by adding a multi-scale frequency transform module and a self-supervised reconstruction auxiliary encoder, outperforming CNN–Transformer hybrids…
desk verdict A clean, incremental Mamba+frequency architecture with a potentially load-bearing ambiguity: the auxiliary 'self-supervised' branch is trained with ground-truth masks and the paper never says what happens at inference. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central objects are the two modules. MSFM projects feature maps into a frequency domain (DWT, FFT, or DCT), applies masks to isolate spectral bands, processes them with scale-adapted convolutions, inverse-transforms, and fuses them back into the spatial feature stream, restoring high-frequency cues that the patch embedding and 1D recurrence suppressed. SSRAE is a Mamba-based auxiliary encoder that shares architecture and parameters with the main segmentation encoder, takes a degraded copy of the input, and is trained to reconstruct the main encoder's high-frequency feature maps, with label-derived binary region masks constraining attention inside each annotated region; a progressive composite loss of L1, cosine similarity, and gradient terms supervises it. The load-bearing physical claim underneath is that the state-space recurrence implements a stable linear dynamical system whose impulse response decays exponentially, so it behaves as a low-pass filter that attenuates high-frequency components during scanning.
What would settle it
Compare FaRMamba against a Mamba baseline augmented with a same-parameter-count convolutional high-frequency enhancement branch: if the Dice gain matches or exceeds FaRMamba's, the low-pass/1D-serialization mechanism is not needed to explain the results; alternatively, compute the high-frequency spectral energy of feature maps before and after the first SS2D block to see whether patch embedding and recurrence actually suppress it.
Extended reading notes
Core claim
On its own terms, the paper's central discovery is a two-part failure analysis and remedy. First, it identifies two concrete deficiencies in Vision Mamba for medical images: local high-frequency capture deficiency (LHICD), caused by patch embedding averaging and the exponential-decay low-pass behavior of the state-space recurrence, and two-dimensional spatial structure degradation (2D-SSD), caused by serializing the image into a 1D causal sequence. Second, it claims both are substantially repaired by MSFM and SSRAE, and that the combination outperforms the compared CNN, transformer, and Mamba baselines on CAMUS, Mouse-cochlea, and Kvasir-Seg, delivering better boundary accuracy, detail preservation, and global coherence without prohibitive computational overhead. The paper also claims a modality-dependent ranking of the three transforms: DWT best on noisy ultrasound, FFT best on MRI because of native k-space sampling, DCT best on textured endoscopy.
Load-bearing premise
The load-bearing premise is that the observed losses of high-frequency detail and spatial structure in Vision Mamba come specifically from patch tokenization and 1D serialization, so that fixing those two mechanisms is what produces the accuracy gains rather than the added parameters or the choice of frequency transform.
Editorial extensions
If this is right
- Vision Mamba encoders can recover boundary detail by injecting frequency-band-restored features, so Mamba-based segmentation need not trade global modeling for fine edges.
- The transform choice matters by modality: use DWT on speckle-heavy ultrasound, FFT on MRI, DCT on textured endoscopy, suggesting modality-aware frequency design.
- A shared-encoder reconstruction auxiliary task improves segmentation without a separate large network or extra labels beyond the segmentation mask.
- MSFM and SSRAE are complementary; ablations show each contributes Dice gains and the full configuration is best.
- The approach extends the same architecture to multiple modalities without redesign, since only the transform changes.
Reading between the lines
- The paper's low-pass story is plausible but not directly measured; one could test it by comparing the high-frequency energy of feature maps before and after Mamba blocks, and by replacing MSFM with an equally sized convolutional high-pass block.
- Because no single transform always wins, a learned soft combination or input-adaptive selection of DWT, FFT, and DCT might beat the fixed per-dataset choice.
- SSRAE uses ground-truth region labels to build attention masks; in low-label regimes, pseudo-labels or unsupervised reconstruction losses could let the same idea work with fewer annotations.
- If the 1D-serialization loss of 2D adjacency is the mechanism, 3D or video Mamba segmentation should suffer similar or larger 2D structure degradation, making the reconstruction aid more valuable there.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FaRMamba, a Vision Mamba variant for medical image segmentation. It argues that Mamba's patch tokenization and 1D serialization cause local high-frequency information deficiency (LHICD) and 2D spatial structure degradation (2D-SSD), and addresses these with two modules: a Multi-Scale Frequency Transform Module (MSFM) that processes features through DWT, FFT, or DCT in separate bands, and a Self-Supervised Reconstruction Auxiliary Encoder (SSRAE) that reconstructs degraded encoder outputs with label-derived region attention. Experiments on Mouse-cochlea (private MRI), CAMUS (echocardiography), and Kvasir-Seg (endoscopy) report Dice and MIoU improvements over CNN, Transformer, and Mamba baselines, with an ablation on CAMUS 2CH attributing gains to both modules. The manuscript also claims modality-dependent superiority of the three frequency transforms.
Significance. If validated, FaRMamba would be a useful and practical frequency-aware augmentation for Mamba-based medical segmentation backbones, with the interesting observation that different frequency transforms suit different imaging modalities. The paper contributes detailed architectural descriptions, ablations of both modules, and evaluations on three datasets, including a public benchmark. The main empirical claim is plausible but not yet established: the SSRAE module is trained using ground-truth segmentation labels with no specified inference-time treatment, which raises a label-leakage concern; the quantitative comparison lacks variance information, statistical tests, and a fixed selection protocol across variants; and the closest frequency-based Mamba competitors cited in Section 2.1 are not included in Table 1. The central derivation is internally consistent, but these load-bearing issues require resolution before the reported gains can be attributed to the proposed mechanism.
major comments (4)
- [Section 3.4, SSRAE] The paper states that 'Binary region masks are first derived from segmentation labels and then resized to match the feature map resolution' and that this mask 'is added directly to the attention logits, thereby constraining attention to remain within each labeled region.' It does not specify how these masks are obtained at inference time, where ground-truth labels are unavailable. Because the Mamba encoder is shared between the segmentation branch and SSRAE, label-derived attention masks can inject region/class information into the encoder's learned representation during training, making the roughly 2.5 DSC gain attributed to 'SSRAE + MSFM' in Table 2 (89.81 vs 87.25) ambiguous evidence for the proposed mechanism. The same section also calls this process 'self-supervised' and says it trains 'without extra labels,' which is contradicted by the use of segmentation labels. Please state the exact inference-time behavior (drop SSRAE, use predicted masks, or another mechanism) and provide an ablation with a label-free version of the region attention.
- [Tables 1 and 2, Section 4.2-4.3] No standard deviations, confidence intervals, or repeated-run information are reported, and many reported gaps are small (e.g., 95.98 vs 95.75 on CAMUS 4CH, 60.89 vs 59.82 on Mouse-cochlea, 88.97 vs 87.39 on Kvasir-Seg). Moreover, the 'Ours' entry is selected per dataset as the best of FFT/DCT/DWT, which inflates the apparent advantage over fixed baselines. The claim that FaRMamba 'consistently outperforms' competitive methods needs means and standard deviations over at least three seeds, a fixed variant or an appropriate multi-comparison correction, and ideally significance tests.
- [Section 2.1 and Section 4.2] The related work discusses GlobalMamba, EM-Net, and P-Mamba as frequency-based Mamba methods, but Table 1 contains no comparison against any of them. Since these are the closest competitors on the same design axis, their omission prevents the reader from assessing whether MSFM provides gains beyond existing frequency-aware Mamba approaches. Please add these baselines under the same training protocol, or explicitly explain why a direct comparison is not possible.
- [Abstract and Section 4.2] The abstract and conclusion claim 'superior boundary accuracy, detail preservation, and global coherence,' but experiments report only DSC and MIoU. No boundary-specific metric (e.g., HD95, boundary F1, or boundary IoU) and no reconstruction-quality metric are provided, so the specific qualitative claims are not directly supported by the reported measurements. Please add such metrics or soften the claims.
minor comments (5)
- [Equation (4)] The DCT formula is missing the v variable in the second cosine factor: it should be cos(pi(2y+1)v/2N), not cos(pi(2y+1)/2N).
- [Throughout] There are several typos and inconsistencies: 'aafrequency' in Section 3.4, 'UMamba' vs 'Umamba' in Table 1 and elsewhere, and 'First,' at the start of the Introduction. A careful proofread is needed.
- [Section 4.1] The Mouse-cochlea dataset is described as private with the placeholder 'xxUniversity'; the institution name and ethical approval details should be provided, and any public release plan stated, to support reproducibility.
- [Table 2] The formatting of Table 2 is difficult to parse: the meaning of the checkmark columns and the reported DSC/MIoU triples should be clarified in the caption or table structure.
- [Section 3.4] The SSRAE description mentions 'multi-head self-attention' even though the architecture is Mamba-based and the main encoder uses SS2D; please clarify whether SSRAE contains attention layers or whether this wording is a typo.
Circularity Check
No significant circularity; the central claims are empirical and benchmarked externally.
full rationale
FaRMamba's contributions (MSFM and SSRAE) are fixed architectural modules evaluated against external benchmarks (CAMUS, Mouse-cochlea, Kvasir-Seg). No predictive equation is fitted to the test data, no fitted parameter is renamed as a prediction, and no load-bearing argument reduces to a self-citation. The motivation that Vision Mamba's patch tokenization and 1D serialization attenuate high-frequency cues and disrupt 2D structure is supported by external references ([29], [35], [38]) and by the paper's own ablations, not by definitional identity with the proposed modules. The SSRAE's label-derived attention masks (Sec. 3.4: 'Binary region masks are first derived from segmentation labels...') raise a legitimate train/test leakage concern that could undermine the attribution of the reported gains, but this is a correctness/validity issue rather than a circular derivation: the modules are not defined in terms of the reported Dice scores, and the results are not statistically forced by construction. The per-dataset choice among DWT/FFT/DCT is a post hoc selection, but the paper does not present it as a prediction derived from an equation. Therefore the circularity burden is low.
Assumptions & free parameters
free parameters (4)
- Gaussian noise standard deviation sigma_noise =
0.01
- Patch size =
4x4
- Number of frequency bands in MSFM =
not specified in text
- Reconstruction loss weighting schedule =
linear decay with EMA smoothing
assumptions (4)
- domain assumption Vision Mamba's state-space recurrence acts as a low-pass filter attenuating high-frequency detail
- domain assumption Frequency-domain transforms can restore or enhance high-frequency cues for segmentation
- domain assumption Auxiliary reconstruction on the same encoder improves the segmentation representation
- standard math Standard definitions of DWT, FFT, and DCT as given in Equations (1) to (4)
Cite this review
Pith. "Pith review of FaRMamba: Frequency-based learning and Reconstruction aided Mamba for Medical Segmentation." pith.science (2026). https://pith.science/paper/5AQBCWMV
@misc{pith2026250720056,
author = {Pith},
title = {Pith review of: FaRMamba: Frequency-based learning and Reconstruction aided Mamba for Medical Segmentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/5AQBCWMV}},
note = {Machine review of arXiv:2507.20056}
}
read the original abstract
Accurate medical image segmentation remains challenging due to blurred lesion boundaries (LBA), loss of high-frequency details (LHD), and difficulty in modeling long-range anatomical structures (DC-LRSS). Vision Mamba employs one-dimensional causal state-space recurrence to efficiently model global dependencies, thereby substantially mitigating DC-LRSS. However, its patch tokenization and 1D serialization disrupt local pixel adjacency and impose a low-pass filtering effect, resulting in Local High-frequency Information Capture Deficiency (LHICD) and two-dimensional Spatial Structure Degradation (2D-SSD), which in turn exacerbate LBA and LHD. In this work, we propose FaRMamba, a novel extension that explicitly addresses LHICD and 2D-SSD through two complementary modules. A Multi-Scale Frequency Transform Module (MSFM) restores attenuated high-frequency cues by isolating and reconstructing multi-band spectra via wavelet, cosine, and Fourier transforms. A Self-Supervised Reconstruction Auxiliary Encoder (SSRAE) enforces pixel-level reconstruction on the shared Mamba encoder to recover full 2D spatial correlations, enhancing both fine textures and global context. Extensive evaluations on CAMUS echocardiography, MRI-based Mouse-cochlea, and Kvasir-Seg endoscopy demonstrate that FaRMamba consistently outperforms competitive CNN-Transformer hybrids and existing Mamba variants, delivering superior boundary accuracy, detail preservation, and global coherence without prohibitive computational overhead. This work provides a flexible frequency-aware framework for future segmentation models that directly mitigates core challenges in medical imaging.
Figures
Reference graph
Works this paper leans on
-
[1]
IEEE Transac- tionsonComputers C-23(1),90–93(1974)
Ahmed, N., Natarajan, T., Rao, K.: Discrete cosine transform. IEEE Transac- tionsonComputers C-23(1),90–93(1974). https://doi.org/10.1109/T-C.1974. 223784
-
[2]
In: Karlinsky, L., Michaeli, T., Nishino, K
Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., Wang, M.: Swin- unet: Unet-like pure transformer for medical image segmentation. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds.) Computer Vision – ECCV 2022 Workshops. pp. 205–218. Springer Nature Switzerland, Cham (2023).https://doi.org/10.1007/ 978-3-031-25066-8_9
work page 2023
-
[3]
Machine Learning28(1), 41–75 (1997)
Caruana, R.: Multitask learning. Machine Learning28(1), 41–75 (1997). https: //doi.org/10.1023/A:1007379606734
-
[4]
In: Linguraru, M.G., Dou, Q., Feragen, A., Giannarou, S., Glocker, B., Lekadir, K., Schnabel, J.A
Chang, A., Zeng, J., Huang, R., Ni, D.: Em-net: Efficient channel and frequency learning with mamba for 3d medical image segmentation. In: Linguraru, M.G., Dou, Q., Feragen, A., Giannarou, S., Glocker, B., Lekadir, K., Schnabel, J.A. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. pp. 266–275. Springer Nature Switzerland, ...
work page 2024
-
[5]
https://doi.org/10.48550/arXiv.2102.04306
Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., Zhou, Y.: Transunet: Transformers make strong encoders for medical image segmentation (2021). https://doi.org/10.48550/arXiv.2102.04306
-
[6]
Medical Image Analysis58, 101539 (2019).https://doi.org/10.1016/j.media
Chen, L., Bentley, P., Mori, K., Misawa, K., Fujiwara, M., Rueckert, D.: Self- supervised learning for medical image analysis using image context restoration. Medical Image Analysis58, 101539 (2019).https://doi.org/10.1016/j.media. 2019.101539
arXiv 2019
-
[7]
https://doi.org/10.48550/arXiv.1412.7062
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected crfs (2016). https://doi.org/10.48550/arXiv.1412.7062
-
[8]
Domain Influence in MRI Medical Image Segmentation: spatial versus k-space inputs
Gösche, E., Eghbali, R., Knoll, F., Rauschecker, A.M.: Domain influence in mri medical image segmentation: Spatial versus k-space inputs (2024).https://doi. org/10.48550/arXiv.2407.01367
work page Pith review arXiv doi:10.48550/arxiv.2407.01367 2024
Show all 42 references
-
[9]
Gu, A.: Modeling sequences with structured state spaces (2023)
2023
- [10]
- [11]
-
[12]
Medical and Biological En- gineering and Computing 42(2), 189–192 (2004)
Gupta, S., Chauhan, R.C., Sexana, S.C.: Wavelet-based statistical approach for speckle reduction in medical ultrasound images. Medical and Biological En- gineering and Computing 42(2), 189–192 (2004). https://doi.org/10.1007/ BF02344630
2004
-
[13]
In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 770–778 (2016).https://doi.org/10.1109/CVPR.2016.90
2016 doi
- [14]
-
[15]
Nature Methods 18(2), 203–211 (2021)
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnu- net: A self-configuring method for deep learning-based biomedical image seg- mentation. Nature Methods 18(2), 203–211 (2021). https://doi.org/10.1038/ s41592-020-01008-z
2021
-
[16]
In: Ro, Y.M., Cheng, W.H., Kim, J., Chu, W.T., Cui, P., Choi, J.W., Hu, M.C., De Neve, W
Jha, D., Smedsrud, P.H., Riegler, M.A., Halvorsen, P., de Lange, T., Johansen, D., Johansen, H.D.: Kvasir-seg: A segmented polyp dataset. In: Ro, Y.M., Cheng, W.H., Kim, J., Chu, W.T., Cui, P., Choi, J.W., Hu, M.C., De Neve, W. (eds.) Mul- tiMedia Modeling. pp. 451–462. Spring...
2020 doi
-
[17]
Physics in Medicine & Biology 69(9), 095022 (2024)
Kofler, A., Wald, C., Kolbitsch, C., V Tycowicz, C., Ambellan, F.: Joint recon- struction and segmentation in undersampled 3d knee mri combining shape knowl- edge and deep learning. Physics in Medicine & Biology 69(9), 095022 (2024). https://doi.org/10.1088/1361-6560/ad3797
2024 doi
-
[18]
Photoacoustics20, 100197 (2020)
Lan, H., Jiang, D., Yang, C., Gao, F., Gao, F.: Y-net: Hybrid deep learning image reconstruction for photoacoustic tomography in vivo. Photoacoustics20, 100197 (2020). https://doi.org/10.1016/j.pacs.2020.100197
2020
-
[19]
IEEE Transactions on Medical Imaging 38(9), 2198–2210 (2019).https://doi.org/10.1109/TMI.2019.2900516
Leclerc, S., Smistad, E., Pedrosa, J., Østvik, A., Cervenansky, F., Espinosa, F., Espeland, T., Berg, E.A.R., Jodoin, P.M., Grenier, T., Lartizien, C., D’hooge, J., Lovstakken, L., Bernard, O.: Deep learning for segmentation using an open large-scale dataset in 2d echocardiogr...
2019
-
[20]
Proceedings of the AAAI Conference on Artificial Intelligence39(5), 4652–4660 (2025)
Li, C., Liu, X., Li, W., Wang, C., Liu, H., Liu, Y., Chen, Z., Yuan, Y.: U-kan makes strong backbone for medical image segmentation and generation. Proceedings of the AAAI Conference on Artificial Intelligence39(5), 4652–4660 (2025). https: //doi.org/10.1609/aaai.v39i5.32491
2025 doi
- [21]
-
[22]
Computers in Biology and Medicine164, 107290 (2023)
Li, P., Zhou, R., He, J., Zhao, S., Tian, Y.: A global-frequency-domain network for medical image segmentation. Computers in Biology and Medicine164, 107290 (2023). https://doi.org/10.1016/j.compbiomed.2023.107290
2023
-
[23]
In: Farkaš, I., Masulli, P., Wermter, S
Li, Y., Wang, Y., Leng, T., Zhijie, W.: Wavelet u-net for medical image segmenta- tion. In: Farkaš, I., Masulli, P., Wermter, S. (eds.) Artificial Neural Networks and Machine Learning – ICANN 2020. pp. 800–810. Springer International Publishing, Cham (2020). https://doi.org/10...
2020 doi
-
[24]
https://doi.org/10.1016/j.media.2021.102035
Ma, J., Chen, J., Ng, M., Huang, R., Li, Y., Li, C., Yang, X., Martel, A.L.: Loss odysseyin medicalimagesegmentation.Medical ImageAnalysis 71,102035 (2021). https://doi.org/10.1016/j.media.2021.102035
2021
- [25]
-
[26]
In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G
Mehta, S., Mercan, E., Bartlett, J., Weaver, D., Elmore, J.G., Shapiro, L.: Y-net: Joint segmentation and classification for diagnosis of breast biopsy images. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) Medical Image Computing a...
-
[27]
IEEE Transactions on Pattern Analysis and Machine Intelligence44(7), 3523–3542 (2022).https://doi.org/10
Minaee, S., Boykov, Y., Porikli, F., Plaza, A., Kehtarnavaz, N., Terzopoulos, D.: Image segmentation using deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence44(7), 3523–3542 (2022).https://doi.org/10. 1109/TPAMI.2021.3059968
2022
- [28]
-
[29]
Oppenheim, A., Verghese, G.: Signals, systems and inference (2017)
2017
-
[30]
Magnetic Resonance in Medicine 42(5), 952– 962 (1999)
Pruessmann, K.P., Weiger, M., Scheidegger, M.B., Boesiger, P.: Sense: Sen- sitivity encoding for fast mri. Magnetic Resonance in Medicine 42(5), 952– 962 (1999). https://doi.org/10.1002/(SICI)1522-2594(199911)42:5<952:: AID-MRM16>3.0.CO;2-S
1999 doi
- [31]
-
[32]
In: Proceedings of the 31st Inter- national Conference on Neural Information Processing Systems
Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Proceedings of the 31st Inter- national Conference on Neural Information Processing Systems. pp. 6000–6010. NIPS’17, Curran Associates Inc., Red Hook, NY...
2017
- [33]
-
[34]
ISPRS Journal of Photogrammetry and Remote Sensing 190, 196–214 (2022).https://doi.org/10.1016/j.isprsjprs.2022.06.008
Wang, L., Li, R., Zhang, C., Fang, S., Duan, C., Meng, X., Atkinson, P.M.: Unet- former: A unet-like transformer for efficient semantic segmentation of remote sens- ing urban scene imagery. ISPRS Journal of Photogrammetry and Remote Sensing 190, 196–214 (2022).https://doi.org/...
2022 doi
-
[35]
Advances in Neural Information Processing Systems 36, 74021–74038 (2023)
Wang, S., Xue, B.: State-space models with layer-wise nonlinearity are universal approximators with exponential decaying memory. Advances in Neural Information Processing Systems 36, 74021–74038 (2023)
2023
-
[36]
Xu, R., Yang, S., Wang, Y., Du, B., Chen, H.: A survey on vision mamba: Mod- els, applications and challenges (2024).https://doi.org/10.48550/arXiv.2404. 18861
2024 doi
- [37]
- [38]
- [39]
-
[40]
In: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI)
Zhou, L., Liu, H., Bae, J., He, J., Samaras, D., Prasanna, P.: Self pre-training with masked autoencoders for medical image classification and segmentation. In: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI). pp. 1–6 (2023). https://doi.org/10.1109/ISBI537...
2023
-
[41]
In: Proceed- ings of the 41st International Conference on Machine Learning
Zhu, L., Liao, B., Zhang, Q., Wang, X., Liu, W., Wang, X.: Vision mamba: Efficient visual representation learning with bidirectional state space model. In: Proceed- ings of the 41st International Conference on Machine Learning. pp. 62429–62442. PMLR, Vienna, Austria (2024)
2024
-
[2018]
pp. 893–901. Springer International Publishing, Cham (2018).https://doi. org/10.1007/978-3-030-00934-2_99
2018 doi
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.