REVIEW 4 major objections 5 minor 18 references
CSFMamba: Cross State Fusion Mamba Operator for Multimodal Remote Sensing Image Classification
T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read The paper claims Mamba's state-space operator can be turned into a multimodal fusion mechanism by cross-feeding each sensor's state parameters, and that the network beats CNN, Transformer, and direct Mamba baselines on HSI-LiDAR classificat
desk verdict A workmanlike Mamba variant for HSI/LiDAR fusion with a genuinely new cross-state mechanism, but the accuracy claims rest on under-specified experimental protocols and small margins. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Cross State Fusion Mamba (CSFM) block. It takes the CLS token learned in each unimodal Mamba encoder, splices it into the other modality's token set, and feeds the combined tokens through a Mamba block in which the input-dependent SSM matrices B and C are generated from the other modality. This mixes the sequence state from one sensor with the contextual parameters of the other, converting the hidden state into a cross-modal fusion variable. The block is iterated over L layers and is paired with a simplified Mamba core that removes the Conv1D and one projection to cut the computational burden.
What would settle it
Run all compared baselines and CSFMamba on a single fixed train/validation split of MUUFL and Houston2018 with identical patch size, augmentation, and sample counts; then run CSFMamba with the cross-state B/C exchange replaced by simple concatenation. If the accuracy margins over MSFMamba and MACN vanish under matched protocols, or if the cross-state-to-concatenation gap disappears, the central superiority claim fails.
Extended reading notes
Core claim
The central claim is that Mamba's selective state space is not only a cheaper replacement for self-attention but a usable fusion operator. Each modality is tokenized with a CLS token that absorbs the modality's global abstract features through a Mamba encoder. Then, in the Cross State Fusion Mamba block, the CLS token of one modality is spliced into the other modality's sequence, and the combined token sequence is processed by a Mamba layer whose B and C parameters are generated from the opposite modality, so the hidden state of HSI is updated under parameters supplied by LiDAR, and vice versa. The authors argue that this intercrossing of internal state spaces resolves Mamba's inability to f
Load-bearing premise
The comparison's load-bearing premise is that every baseline was trained and validated under the same protocol; the paper reports patch size, batch size, optimizer, and iteration count but not the train/validation split ratio or per-class sample counts, so reported gains could in principle be protocol effects rather than architecture effects.
Editorial extensions
If this is right
- On the two reported datasets, Mamba-based cross-state fusion reaches higher OA, AA, and kappa than CNN, Transformer, and the direct multimodal Mamba baseline, with 0.219M parameters and 12.29M FLOPs.
- Ablations show each design element contributes: removing the preprocessing stage drops OA from 91.13% to 89.68%, removing the CNN feature extraction drops it to 88.61%, and removing the cross-state block drops it to 82.94%.
- Because the SSM formulation is linear in sequence length, the fusion layer's cost scales linearly with image patches rather than quadratically, which is the paper's main reason for expecting scalability to larger scenes.
- The classification output averages the CLS tokens of both modalities, so the final decision is explicitly forced to use fused information rather than a single modality's features.
Reading between the lines
- The cross-state trick is not obviously limited to two sensors: a chain of CLS-token handoffs among HSI, LiDAR, SAR, or other spectral data is a natural extension, though the paper does not test it.
- One direct experiment to isolate the mechanism would be to keep token splicing but disable the cross-generation of B and C, forcing fusion to happen only through the CLS token; the accuracy gap would show how much of the gain comes from state-parameter exchange rather than token pooling.
- A matched train/validation split and per-class sample counts for every baseline would let a reader separate architecture gains from protocol effects, since the paper specifies only patch size, batch size, optimizer, and iteration count.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CSFMamba, a multimodal fusion network for HSI/LiDAR remote sensing image classification. The architecture combines CNN-based local feature extraction with a simplified Mamba encoder, and introduces a Cross State Fusion Mamba block in which CLS tokens and the SSM parameter matrices B and C are exchanged between modalities to achieve cross-modal state-space fusion. Experiments on MUUFL and Houston2018 report state-of-the-art overall accuracy, average accuracy, and kappa, while using 0.219M parameters and 12.29M FLOPs. The paper also includes an ablation study and a brief burden analysis.
Significance. If the reported results are reproducible, the cross-state fusion idea is a genuinely interesting adaptation of SSM/selective state-space models to multimodal fusion, and the linear complexity claim is attractive for large remote sensing scenes. The paper explicitly targets a known limitation of plain Mamba—its inability to directly perform feature fusion—and proposes a concrete architectural remedy. However, the evidence supporting the central superiority claim is currently weakened by a poorly specified experimental protocol, single-run results, and unverified baseline comparisons.
major comments (4)
- [Section III.A and Tables I-II] The experimental protocol is underspecified. Section III.A gives dataset dimensions and hyperparameters but omits the train/validation/test split ratio, the number of labeled samples per class, and the patch sampling strategy. Table I and Table II report single-run OA/AA/Kappa with no standard deviations or significance tests, and all baseline numbers are imported from prior publications. The claimed margins are small: +0.47 percentage points over MACN on MUUFL and +1.00 over MSFMamba on Houston2018. Under a different split or with run-to-run variation, these margins could vanish. The authors should specify the exact data split, report results over multiple random seeds (mean±std), and either reimplement baselines under the same protocol or provide a matched-protocol comparison. This is load-bearing for the paper's main claim of superiority.
- [Section III.A, 'training process has 200 iterations'] The text says there are 200 iterations in total with batch size 256. Taken literally, this is only about 51,200 training samples, which is a very small fraction of the Houston2018 image (1202×4768 pixels). This seems inconsistent with the high reported accuracy and suggests the term 'iterations' may mean 'epochs' or that a much smaller training set is used than the reader would expect. The authors must clarify the training schedule and dataset partition. This detail directly affects whether the comparisons with baselines trained under different schedules are meaningful.
- [Table III] The burden analysis is not anchored to a concrete protocol. FLOPs and parameter counts depend on input tensor size, patch size, sequence length, and implementation details. The table reports values for CoupledCNN, FGCN, HCT, MACN, and Ours, but does not state the input size or the FLOPs calculation method for each model. The claim that 'the training and testing time is shorter' is also unsupported by any timing measurement. To make the complexity advantage verifiable, the authors should specify the exact input configuration and, ideally, provide measured runtime on the same hardware.
- [Section II.D, Eq.(7)] The cross-state fusion mechanism needs a precise formulation. In Eq.(7), B and C are generated from X^cross_L,t while the state update uses X^cross_H,t. The dimensions and ordering of these operations are not defined: it is unclear whether Eq.(7) follows Eq.(6) directly, whether the same procedure is applied symmetrically for the LiDAR branch, and how the two modalities' states are interleaved across the L layers. Additionally, B is used in Eq.(1) as an output dimension and in Eqs.(3)-(7) as an SSM parameter; this overloading makes the derivation hard to follow. Please clarify the shapes and provide a precise pseudocode or diagram of the fusion-SSM.
minor comments (5)
- [Equations (3)-(4)] Equation (4) uses (∆A)^{-1}(exp(∆A)-I), which is the standard zero-order hold discretization but can be numerically unstable for small ∆A. In practice this is often computed with a scaled matrix exponential; please clarify whether a numerically stable implementation is used.
- [Eq.(1) and Eq.(5)] The symbol B is used for both the linear-layer output dimension and the SSM input matrix. Use a different symbol (e.g., d_model) for the dimension to avoid ambiguity.
- [Table III] The unit 'FLOPs(M)' is ambiguous; it should be written as 'MFLOPs' or '×10^6 FLOPs' for consistency with common usage.
- [Section II.C] The paper says CSFMamba 'simplifies the structure of Mamba by removing Conv1D and a projection function.' This design choice is not justified or ablated. It would be useful to include an ablation variant that retains these components, since removing them may affect the reported performance.
- [General] No code or data availability statement is provided. Given the protocol sensitivity of the results, releasing code and exact split indices would substantially strengthen reproducibility.
Circularity Check
No significant circularity: CSFMamba's contribution is an architectural construction evaluated on held-out test sets; no equation or fitted parameter reduces to the claimed result.
full rationale
The paper's derivation chain is self-contained in the relevant sense. The proposed CSFMamba network is defined by concrete architectural operations: PCA and mutual-information preprocessing for HSI, CNN feature extraction, tokenization with CLS tokens, a simplified Mamba encoder, and a cross-state fusion block expressed by Eqs. (6)–(8). These are constructive definitions, not fitted parameters relabeled as predictions. The central claim—superior OA/AA/Kappa on MUUFL and Houston2018—is an empirical result obtained by training and testing the model, not a quantity derived from the model's own inputs by construction. The ablation study (Table IV) removes modules and reports accuracy changes, which is a causal test rather than a circular reduction. The comparison baselines are drawn from prior external papers (EndNet, CoupledCNN, FGCN, HCT, MACN, MSFMamba); even if those numbers were produced under different protocols, protocol mismatch is an experimental-validity concern, not circularity. References [3] and [4] are prior work by the same research group, but they are cited only as background on multimodal fusion and are not load-bearing for the architecture's novelty or for the reported accuracy. No uniqueness theorem, ansatz, or self-citation chain is invoked to force the design choice. Thus no circular step meets the evidentiary bar of Eq. X = Eq. Y by construction or fitted-parameter-renamed-as-prediction.
Assumptions & free parameters
free parameters (10)
- patch size s =
11
- initial learning rate =
5e-4
- learning rate decay =
0.5
- batch size =
256
- training iterations =
200
- number of Mamba layers L =
not reported
- hidden/feature dimension D =
not reported
- SSM state size N =
not reported
- number of PCA components C1 =
not reported
- LiDAR derived feature channels C2 =
not reported
assumptions (5)
- standard math Standard zero-order hold discretization of continuous SSM
- domain assumption Mamba's selective SSM is differentiable and its linear-time complexity claim holds for image tokens
- domain assumption CLS token contains a complete modality-level summary after Mamba encoding
- ad hoc to paper Cross-state parameter generation with the other modality's tokens produces semantically meaningful fusion
- domain assumption CNN local feature extraction improves Mamba sequence input
Cite this review
Pith. "Pith review of CSFMamba: Cross State Fusion Mamba Operator for Multimodal Remote Sensing Image Classification." pith.science (2026). https://pith.science/paper/DH3EUDDR
@misc{pith2026250900677,
author = {Pith},
title = {Pith review of: CSFMamba: Cross State Fusion Mamba Operator for Multimodal Remote Sensing Image Classification},
year = {2026},
howpublished = {\url{https://pith.science/paper/DH3EUDDR}},
note = {Machine review of arXiv:2509.00677}
}
read the original abstract
Multimodal fusion has made great progress in the field of remote sensing image classification due to its ability to exploit the complementary spatial-spectral information. Deep learning methods such as CNN and Transformer have been widely used in these domains. State Space Models recently highlighted that prior methods suffer from quadratic computational complexity. As a result, modeling longer-range dependencies of spatial-spectral features imposes an overwhelming burden on the network. Mamba solves this problem by incorporating time-varying parameters into ordinary SSM and performing hardware optimization, but it cannot perform feature fusion directly. In order to make full use of Mamba's low computational burden and explore the potential of internal structure in multimodal feature fusion, we propose Cross State Fusion Mamba (CSFMamba) Network. Specifically, we first design the preprocessing module of remote sensing image information for the needs of Mamba structure, and combine it with CNN to extract multi-layer features. Secondly, a cross-state module based on Mamba operator is creatively designed to fully fuse the feature of the two modalities. The advantages of Mamba and CNN are combined by designing a more powerful backbone. We capture the fusion relationship between HSI and LiDAR modalities with stronger full-image understanding. The experimental results on two datasets of MUUFL and Houston2018 show that the proposed method outperforms the experimental results of Transformer under the premise of reducing the network training burden.
Figures
Reference graph
Works this paper leans on
-
[1]
Spectral-spatial latent recon- struction for open-set hyperspectral image classification,
J. Yue, L. Fang, and M. He, “Spectral-spatial latent recon- struction for open-set hyperspectral image classification,” IEEE Transactions on Image Processing , vol. 31, pp. 5227–5241, 2022
work page 2022
-
[2]
N. Jiang, H.-B. Li, C.-J. Li, H.-X. Xiao, and J.-W. Zhou, “A fusion method using terrestrial laser scanning and unmanned aerial vehicle photogrammetry for landslide deformation moni- toring under complex terrain conditions,” IEEE Transactions on Geoscience and Remote Sensing , vol. 60, pp. 1–14, 2022
work page 2022
-
[3]
Robust land cover classification with multimodal knowledge distillation,
G. Xu, X. Jiang, Y . Zhou, S. Li, X. Liu, and P. Lin, “Robust land cover classification with multimodal knowledge distillation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1–16, 2024
work page 2024
-
[4]
G. Xu, X. Jiang, X. Li, Z. Zhang, and X. Liu, “Exploring self- supervised learning for multi-modal remote sensing pre-training via asymmetric attention fusion,” Remote Sensing , vol. 15, no. 24, p. 5682, 2023
work page 2023
-
[5]
Classification of hyperspectral remote sensing images with support vector machines,
F. Melgani and L. Bruzzone, “Classification of hyperspectral remote sensing images with support vector machines,” IEEE Transactions on Geoscience and Remote Sensing, vol. 42, no. 8, pp. 1778–1790, 2004
work page 2004
-
[6]
Hy- perspectral image classification—traditional to deep models: A survey for future prospects,
M. Ahmad, S. Shabbir, S. K. Roy, D. Hong, X. Wu, J. Yao, A. M. Khan, M. Mazzara, S. Distefano, and J. Chanussot, “Hy- perspectral image classification—traditional to deep models: A survey for future prospects,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , vol. 15, pp. 968–999, 2022
work page 2022
-
[7]
Deep encoder–decoder networks for classification of hyperspectral and lidar data,
D. Hong, L. Gao, R. Hang, B. Zhang, and J. Chanussot, “Deep encoder–decoder networks for classification of hyperspectral and lidar data,” IEEE Geoscience and Remote Sensing Letters , vol. 19, pp. 1–5, 2022
work page 2022
-
[8]
Classification of hyperspectral and lidar data using coupled cnns,
R. Hang, Z. Li, P. Ghamisi, D. Hong, G. Xia, and Q. Liu, “Classification of hyperspectral and lidar data using coupled cnns,” IEEE Transactions on Geoscience and Remote Sensing , vol. 58, no. 7, pp. 4939–4950, 2020
work page 2020
Show all 18 references
-
[9]
Fractional ga- bor convolutional network for multisource remote sensing data classification,
X. Zhao, R. Tao, W. Li, W. Philips, and W. Liao, “Fractional ga- bor convolutional network for multisource remote sensing data classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–18, 2022
2022
-
[10]
Convolutional recurrent neural networks for hyperspectral data classification,
H. Wu and S. Prasad, “Convolutional recurrent neural networks for hyperspectral data classification,” Remote Sensing , vol. 9, no. 3, p. 298, 2017
2017
-
[11]
Attention is all you need,
A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems , 2017
2017
-
[12]
Joint classification of hyperspectral and lidar data using a hierarchical cnn and transformer,
G. Zhao, Q. Ye, L. Sun, Z. Wu, C. Pan, and B. Jeon, “Joint classification of hyperspectral and lidar data using a hierarchical cnn and transformer,” IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–16, 2023
2023
-
[13]
Mixing self-attention and convolution: A unified framework for multi- source remote sensing data classification,
K. Li, D. Wang, X. Wang, G. Liu, Z. Wu, and Q. Wang, “Mixing self-attention and convolution: A unified framework for multi- source remote sensing data classification,” IEEE Transactions on Geoscience and Remote Sensing , vol. 61, pp. 1–16, 2023
2023
-
[14]
Mamba: Linear-time sequence modeling with selective state spaces,
A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023
2023 arXiv
-
[15]
Efficiently modeling long sequences with structured state spaces,
A. Gu, K. Goel, and C. R ´e, “Efficiently modeling long sequences with structured state spaces,” arXiv preprint arXiv:2111.00396, 2021
2021 arXiv
-
[16]
Hsimamba: Hyperpsectral imaging efficient feature learning with bidirectional state space for classification,
J. X. Yang, J. Zhou, J. Wang, H. Tian, and A. W. C. Liew, “Hsimamba: Hyperpsectral imaging efficient feature learning with bidirectional state space for classification,” arXiv preprint arXiv:2404.00272, 2024
2024 arXiv
-
[17]
Spectral-spatial mamba for hyperspectral image classification,
L. Huang, Y . Chen, and X. He, “Spectral-spatial mamba for hyperspectral image classification,” arXiv preprint arXiv:2404.18401, 2024
2024 arXiv
-
[18]
Msf- mamba: Multi-scale feature fusion state space model for multi- source remote sensing image classification,
F. Gao, X. Jin, X. Zhou, J. Dong, and Q. Du, “Msf- mamba: Multi-scale feature fusion state space model for multi- source remote sensing image classification,” arXiv preprint arXiv:2408.14255, 2024
2024 arXiv
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.