{"id":"fdf5f478-4102-4343-8453-a2df3a42645b","arxiv_id":"2502.00435","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SatMamba shows that a Mamba-based masked autoencoder matches ViT-based MAE on remote sensing segmentation and damage assessment, with efficiency linear in sequence length only at larger inputs.","lead":"This paper introduces SatMamba, a self-supervised pretraining framework that uses Mamba state space models instead of vision transformers in a masked autoencoder. On two remote sensing tasks it matches ViT-based pretraining, but only with twice as many parameters at the tested image size.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Linear-scaling advantage is asserted from resource curves, not tested on long sequences; all experiments are 224x224 RGB where SatMamba is slower.","rationale":"The reader's weakest assumption identifies the same gap: the paper's motivation is linear scaling for long sequences, but all experiments are short RGB sequences where SatMamba is less efficient. This is load-bearing because the novelty of the work rests on the claim that Mamba-based MAE can match ViT-based MAE while scaling better. If the linear-scaling advantage is not demonstrated, the contribution reduces to a minor architectural variant with no demonstrated benefit. The paper itself acknowledges the higher initial cost and memory issues, and even states that multispectral/temporal evaluation is future work (Section III-A.1). The parameter mismatch (229.86M vs. 111.66M) further weakens the performance comparison, but the central unresolved issue is the missing long-sequence evidence. A concrete long-sequence downstream test with matched training protocols and seeds would settle whether the claimed advantage actually materializes. Since the reader's conditional verdict already hinges on exactly this concern, my read does not change the verdict.","tokens_in":9181,"tokens_out":6011,"duration_ms":53758,"concrete_test":"Fine-tune SatMamba-B and ViTMAE-B on the xBD dataset at original 1024x1024 resolution (L=4096 patches with patch size 16) for building damage assessment, using at least 3 seeds and reporting wall-clock time, peak GPU memory, and F1 scores. If SatMamba does not show a favorable compute/memory crossover at this long sequence while maintaining F1 within noise of ViTMAE-B, the linear-scaling advantage is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SatMamba offers linear-scaling advantages for long remote-sensing sequences is not supported by any experiment in the paper. All pretraining and downstream tasks use 224x224 RGB inputs (Section IV-A, Tables II and III), where SatMamba-B consumes about twice the compute and more GPU memory than ViTMAE-B (Section IV-A, Figure 3). The paper explicitly defers multispectral and multitemporal evaluation to future work (Section III-A.1). The only evidence for the linear-scaling benefit is a resource curve (Figure 3) extrapolated to larger inputs, not a downstream test at long sequence lengths. Moreover, the performance comparison is confounded by parameter count: SatMamba-B has 229.86M parameters vs. 111.66M for ViTMAE-B (Table I), so 'competitive' results may reflect capacity rather than the Mamba architecture itself. Without a parameter-matched or long-sequence experiment, the two-part central claim (competitive performance plus linear advantage) is unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"SatMamba proposes a masked-autoencoder pretraining architecture for remote sensing imagery in which the ViT encoder/decoder blocks are replaced by multi-directional Mamba (state space) blocks. The method is pretrained on fMoW RGB images at 224x224 resolution and evaluated on two image-to-image downstream tasks: semantic segmentation on OpenEarthMap and building damage assessment on xBD. The paper reports that SatMamba-B without positional encodings achieves an mIoU of 66.46% on OpenEarthMap and an F1 score of 80.55% on xBD, slightly outperforming ViTMAE-B (65.62% mIoU, 79.45% F1) and EfficientNet-B7 (66.34% mIoU, 80.08% F1). The authors argue that SatMamba's linear computational scaling makes it particularly advantageous for long input sequences such as multispectral and multitemporal remote sensing data, while acknowledging that at the tested 224x224 resolution it consumes about twice the compute and more memory than ViTMAE-B.","tokens_in":9434,"tokens_out":3231,"duration_ms":30505,"significance":"If the central claims were fully supported, the paper would make a useful contribution: it would demonstrate that a state-space-model backbone can be integrated into an MAE pretraining framework for remote sensing, with downstream performance competitive with ViT-based MAEs and CNNs, plus a theoretical linear-scaling advantage for long sequences. The paper has several strengths: it provides source code, it honestly reports the parameter increase and higher initial cost of SatMamba, it includes CNN baselines that are often missing in remote sensing foundation model papers, it performs ablations over scanning directions and positional encodings, and it evaluates on two distinct downstream tasks. The main weakness is that the experiments do not actually exercise the long-sequence regime that motivates the architecture, and the performance comparison is confounded by a 2x parameter/compute gap. The work is therefore incremental but potentially valuable if the claims are either strengthened with additional experiments or appropriately narrowed.","major_comments":[{"comment":"The comparison is not parameter-matched. SatMamba-B has 229.86M parameters versus 111.66M for ViTMAE-B, and at the 224x224 resolution used in all experiments it consumes roughly twice the compute and more GPU memory than ViTMAE-B, as the paper itself states. The paper also notes that the increased parameter count was necessary to achieve pretraining results comparable to ViTMAE-B. Consequently, the reported downstream gains (Tables II and III) may reflect the additional capacity or compute rather than the Mamba architecture itself. A parameter-matched or compute-matched comparison is needed to support the claim that SatMamba is a viable alternative to ViT-based pretraining.","section":"Section IV-A, Table I, Figure 3"},{"comment":"The central motivating advantage, linear scaling for long remote-sensing sequences, is not tested. All pretraining and downstream experiments use 224x224 RGB images; the paper explicitly defers multispectral and multitemporal evaluation to future work. Figure 3 is a resource curve extrapolated to larger input sizes, not a measurement of downstream accuracy or speed at long sequence lengths. The conclusion that SatMamba 'enables efficient self-supervised learning' for such data is therefore unsupported. Either add a long-sequence or multiband experiment, or revise the conclusion to state that linear scaling is a theoretical property that remains to be demonstrated.","section":"Section III-A.1 and Section IV-A"},{"comment":"Results are reported from single runs without variance estimates. The gaps between SatMamba-B w/o pos and ViTMAE-B are 0.84 mIoU points (66.46 vs 65.62) and 1.10 F1 points (80.55 vs 79.45). In the absence of multiple seeds or statistical testing, these small margins could be within run-to-run variability, which weakens the 'competitive' claim. Please report mean and standard deviation over at least three seeds, or provide a statistical significance test.","section":"Section IV-B, Tables II and III"}],"minor_comments":[{"comment":"The text says the tensor has shape R(H/P)x(H/P)xD, but it should presumably be R(H/P)x(W/P)xD; the width dimension is missing.","section":"Section III-A.2"},{"comment":"Figure 1 is never explicitly referenced in the method text; please add a reference when the encoder and decoder are first described.","section":"Figure 1"},{"comment":"The author affiliation sentence reads 'Hiromichi Fukui are with'; it should be 'Hiromichi Fukui is with'.","section":"Author block"},{"comment":"The evaluation protocol is attributed to reference [11], a change detection paper; please verify that this is the correct source for the xBD evaluation metrics.","section":"Section IV-B.2"},{"comment":"The discussion of ViTMAE-B outperforming ViTMAE-L mentions pretraining resolution and model size as possible causes, but this confound is not controlled; it would be helpful to acknowledge this limitation in the discussion.","section":"Section IV-B.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and honest about its limitations, but the novelty is incremental: it applies an existing multi-directional SSM scanning design to an MAE pretraining framework for RGB high-resolution remote sensing data, and the evaluation does not yet demonstrate the claimed long-sequence advantage. The main issues are load-bearing and can be addressed either by adding parameter-matched or long-sequence experiments or by narrowing the claims in the abstract and conclusion. Given the journal's scope, a revised version with these changes would be suitable; in its current form the central selling point is not yet verified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look, but keep expectations calibrated. The genuinely new thing here is the first combination, in the cited literature, of masked autoencoding with a Mamba backbone for high-resolution RGB remote sensing, including a four-scan-direction design and a positional-encoding ablation. The paper ships code, the ablations are honest, and the authors openly state that SatMamba costs twice as much as ViTMAE-B at 224x224 and that multispectral/multitemporal evaluation is future work. That transparency is real credit.\n\nWhat the paper actually shows: SatMamba-B w/o pos edges out ViTMAE-B on OpenEarthMap mIoU (66.46 vs 65.62) and on xBD overall F1 (80.55 vs 79.45). Those are modest gains, and EfficientNet-B7 is within a hair of both on both tasks, which matches the emerging picture that CNN baselines are not obsolete. The pretraining loss curves also look plausible, with SatMamba slightly lower than ViTMAE-B after 800 epochs.\n\nNow the soft spots, in proportion. The central efficiency claim—linear scaling, advantageous for long sequences—is not demonstrated by any downstream experiment. Every test uses 224x224 RGB, where SatMamba is slower and heavier. The linear-scaling evidence is a resource curve extrapolated beyond tested sizes. That is a real gap, but the paper does not hide it; it is flagged in the text and in the conclusion. The parameter mismatch is also real: SatMamba-B has 229.86M params vs 111.66M for ViTMAE-B, so the performance comparison is capacity-confounded. Single-run results with no error bars make it hard to know if the 0.8–1.1 point gaps are meaningful. These are fixable with parameter-matched or compute-matched comparisons, long-sequence tests (even synthetic), and repeated runs.\n\nOne thing the stress-test note gets slightly wrong: the architecture does have linear complexity as a built-in property, so the theoretical claim is not fabricated. The issue is that the practical benefit for remote sensing is unverified. I would not call this a load-bearing flaw; I would call it an overreach in the abstract and conclusion relative to the evidence. The body is more careful.\n\nWho is this for? Researchers working on Mamba-based vision backbones or geospatial foundation models. It is a useful data point, not a breakthrough. A serious referee should see it, because the architecture combination is plausible and the empirical setup is reproducible enough to improve.\n\nRecommendation: accept for peer review, but with a clear request for parameter-matched comparisons, long-sequence evidence, and uncertainty quantification before publication.","headline":"SatMamba is a legitimate first step—MAE pretraining with a Mamba backbone for high-res RGB—but the linear-scaling selling point rests on resource curves, not on any long-sequence experiment.","tokens_in":9900,"tokens_out":1669,"would_cite":true,"duration_ms":17373,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Mamba in a masked autoencoder matches ViT on remote sensing tasks","keywords":["masked autoencoder","state space models","Mamba","remote sensing","semantic segmentation","building damage assessment","self-supervised pretraining","linear complexity"],"falsifier":"Fine-tune SatMamba-B and ViTMAE-B on inputs with genuinely long sequences, for example $512\\times512$ crops, 10-band multispectral images, or six-frame temporal stacks, and compare wall-clock training time, peak memory, and task accuracy. If SatMamba consumes more compute and memory at those sizes while only matching accuracy, the paper's central efficiency-and-performance claim fails.","tokens_in":1674,"feed_emoji":"🛰️","tokens_out":5748,"duration_ms":82612,"temperature":0.7,"pith_summary":"This paper tries to show that a state-space-model backbone, Mamba, can replace the Vision Transformer inside a masked autoencoder for remote sensing pretraining, with no loss of downstream accuracy and with computational cost that grows linearly rather than quadratically with input length. The authors pretrain SatMamba on 363,572 RGB images from the fMoW dataset and fine-tune it on two image-to-image tasks: land-cover segmentation on OpenEarthMap and building damage assessment on xBD. On both tasks the best SatMamba variant edges out the ViT-based baselines and the strong CNN baselines. The efficiency argument is conditional: at the $224\\times224$ RGB resolution used in all experiments SatMamba costs about twice the compute of the ViT baseline, and the linear-scaling benefit only appears as sequence length grows.","feed_headline":"Mamba-based pretraining beats ViT on two remote sensing tasks","feed_subtitle":"SatMamba tops ViT baselines on OpenEarthMap and xBD, with linear scaling promised for longer inputs.","key_machinery":"The central object is the multi-way Mamba block: a layer containing several Mamba blocks, one per scanning direction, whose outputs are merged. Mamba is a selective state space model that maps a sequence $x(t)$ to $y(t)$ through a hidden state $h(t)$ with $h'(t)=Ah(t)+Bx(t)$ and $y(t)=Ch(t)+Dx(t)$, discretized with a timescale parameter; the paper uses Mamba 2. Four scanning directions let the model build a 2D-aware representation despite the 1D sequential scan. The encoder applies masking to the input after patchification and before flattening, keeps the natural patch order, and optionally adds fixed positional encodings. This design is what connects the masked autoencoder objective to a linear-complexity sequence model.","core_discovery":"SatMamba is a masked autoencoder whose encoder and decoder are built from multi-way Mamba blocks instead of transformer blocks. Masking is applied to the patch grid before flattening, the flattened sequence keeps its natural spatial order, and each layer scans the tokens in up to four directions (row-major forward and backward, column-major forward and backward) before merging. With these design choices, SatMamba-B without positional encodings reaches 66.46% mean IoU on OpenEarthMap and an 80.55% overall F1 score on xBD, the best results among ResNet50, EfficientNet-B7, ViTMAE-B, and ViTMAE-L; it also achieves lower pretraining reconstruction loss than ViTMAE-B. The paper claims this makes Mamba a viable backbone for remote sensing foundation models, with linear computational scaling that should benefit long sequences such as multispectral, multitemporal, and hyperspectral inputs, while acknowledging higher initial cost at small input sizes.","pith_inferences":["Beyond the paper's experiments, the linear-scaling advantage remains extrapolation: every downstream test runs at $224\\times224$ RGB, so a fair comparison at $512\\times512$ crops, 10-band multispectral images, or six-frame temporal stacks is needed to confirm that SatMamba's efficiency claim holds where ViT becomes expensive.","The result that positional encodings hurt SatMamba suggests that for state-space vision models the scan order may already provide enough positional information, a hypothesis worth testing on natural images and other SSM-based vision architectures.","Because SatMamba preserves natural patch order, it cannot exploit the random-shuffle ordering that MAE uses as a form of augmentation; a variant that shuffles masked tokens might either widen or close the gap with ViTMAE, and the paper does not test it.","If the linear scaling holds at long sequence lengths, SatMamba could process full multispectral cubes and temporal stacks directly, avoiding the spectral-group patching and pooling tricks currently used to keep ViT sequence lengths manageable."],"forward_implications":["SatMamba can serve as a pretraining backbone for high-resolution RGB remote sensing, matching or exceeding ViT-based masked autoencoders on semantic segmentation and building damage assessment.","Because its cost scales linearly with input length, SatMamba should scale better than ViT on the long input sequences typical of multispectral, multitemporal, and hyperspectral data.","Removing fixed positional encodings improves SatMamba's fine-tuned accuracy, indicating that the sequential state itself carries spatial order information.","The architecture can be adapted to other image domains, including medium-resolution, multispectral, and natural images, since it relies only on standard patchification and masking.","Pretrained Mamba-based encoders can be dropped into a UNet-style fine-tuning pipeline for image-to-image tasks."],"supporting_citations":[{"why":"Supplies the masked autoencoder pretraining objective and the normalized-pixel reconstruction target that SatMamba adapts.","marker":"[5]"},{"why":"Introduces the Mamba selective state space model that provides the linear-time sequence backbone.","marker":"[7]"},{"why":"Presents Mamba 2, the multi-headed state space implementation used in SatMamba's blocks.","marker":"[14]"},{"why":"Motivates bidirectional row-major scanning for vision Mamba, which SatMamba extends to four directions.","marker":"[9]"},{"why":"Shows four-direction scanning for visual state space models, the basis of SatMamba's multi-way block.","marker":"[10]"},{"why":"Defines the SatMAE pretraining setup and the ViTMAE-L baseline that SatMamba compares against.","marker":"[1]"},{"why":"Provides the fMoW dataset used for all pretraining experiments.","marker":"[19]"},{"why":"Supplies the OpenEarthMap dataset and eight land-cover labels for the semantic segmentation fine-tuning evaluation.","marker":"[21]"},{"why":"Provides the xBD dataset and the xView2 evaluation protocol for building damage assessment.","marker":"[18]"}],"fun_headline_variants":["SatMamba: Mamba backbone beats ViT on two remote sensing tasks","Linear-scaling Mamba tops ViT on OpenEarthMap and xBD","Mamba + masked autoencoder outdoes ViTMAE on satellite data","State Space Model foundation model rivals ViT for Earth observation","Mamba-based pretraining with linear scaling wins on remote sensing"],"cache_read_input_tokens":12160,"weakest_assumption_plain":"The load-bearing premise is that Mamba's linear-in-sequence-length complexity will translate into a practical efficiency advantage on the long, multiband, or multitemporal satellite inputs where ViT becomes expensive; the paper's own experiments only exercise $224\\times224$ RGB images, where SatMamba actually costs more.","fun_headline_variants_meta":{"raw":{"variants":["SatMamba: Mamba backbone beats ViT on two remote sensing tasks","Linear-scaling Mamba tops ViT on OpenEarthMap and xBD","Mamba + masked autoencoder outdoes ViTMAE on satellite data","State Space Model foundation model rivals ViT for Earth observation","Mamba-based pretraining with linear scaling wins on remote sensing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1428,"prompt_tokens":987,"completion_tokens":441,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":347}},"tokens_in":603,"tokens_out":441,"duration_ms":4644,"temperature":1.0,"reasoning_tokens":347,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T18:59:56.015481+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fine-tune SatMamba-B and ViTMAE-B on inputs with genuinely long sequences, for example $512\\times512$ crops, 10-band multispectral images, or six-frame temporal stacks, and compare wall-clock training time, peak memory, and task accuracy. If SatMamba consumes more compute and memory at those sizes while only matching accuracy, the paper's central efficiency-and-performance claim fails.","supporting_citations":[{"cited_title":"Masked Autoencoders Are Scalable Vision Learners,","cited_arxiv_id":null,"evidence_quote":"Supplies the masked autoencoder pretraining objective and the normalized-pixel reconstruction target that SatMamba adapts."},{"cited_title":"Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model,","cited_arxiv_id":null,"evidence_quote":"Motivates bidirectional row-major scanning for vision Mamba, which SatMamba extends to four directions."},{"cited_title":"SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery,","cited_arxiv_id":null,"evidence_quote":"Defines the SatMAE pretraining setup and the ViTMAE-L baseline that SatMamba compares against."},{"cited_title":"Functional Map of the World,","cited_arxiv_id":null,"evidence_quote":"Provides the fMoW dataset used for all pretraining experiments."},{"cited_title":"OpenEarthMap: A Benchmark Dataset for Global High-Resolution Land Cover Map- ping,","cited_arxiv_id":null,"evidence_quote":"Supplies the OpenEarthMap dataset and eight land-cover labels for the semantic segmentation fine-tuning evaluation."},{"cited_title":"Creating xbd: A dataset for assessing building damage from satellite imagery,","cited_arxiv_id":null,"evidence_quote":"Provides the xBD dataset and the xView2 evaluation protocol for building damage assessment."}],"review_version":1}