Pith. sign in

REVIEW 5 major objections 4 minor 24 references

SAMST: A Transformer framework based on SAM pseudo label filtering for remote sensing semi-supervised semantic segmentation

T0 review · 5 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read SAMST claims that a frozen SAM can be used to clean noisy pseudo-labels, lifting semi-supervised segmentation on Potsdam to 70.80 mIoU from a 67.00 baseline with only 1/32 of labels.

desk verdict Plausible SAM-based pseudo-label refinement recipe, but Table I is internally inconsistent and the evidence is under-specified. read the letter →

arxiv 2507.11994 v1 pith:GETPIG2J submitted 2025-07-16 cs.CV

classification cs.CV
keywords semi-supervisedsemanticsegmentationremotesensingpseudo-labelrefinementSegmentAnythingModelself-traininglandcovertransformerPotsdamdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SAMST claims that a frozen Segment Anything Model (SAM) can be used as a post-hoc refiner to clean the noisy pseudo-labels that a weakly supervised segmentation model produces, and that this lets semi-supervised remote sensing segmentation work well with just 1/32 of the training data labeled. On the Potsdam dataset, the method reports mIoU 70.80, OA 86.44, and mF1 81.65 after a single self-training iteration, beating its own supervised baseline by 5.67 mIoU points and outperforming three published semi-supervised methods. The paper argues that the gain comes from improving pseudo-label accuracy rather than from more training, because the refiner removes low-confidence predictions, prompts SAM to redraw class-agnostic object boundaries, and stitches the corrected masks back into the label map with probability-based thresholds. A composite weighted loss that downweights pseudo-labeled samples is introduced to keep label noise from derailing the model.

What carries the argument

The load-bearing mechanism is the SAM-based Pseudo-label Refiner, a three-module pipeline: (1) the Threshold Filter Module zeroes out low-confidence predictions into an ignored label (255); (2) the Prompt Generation Module converts the filtered map into connected regions, computes maximum bounding rectangles expanded by Bn pixels, and samples positive/negative points with proximity and probability constraints to prompt SAM; (3) the Label Refinement Module takes SAM's masks, removes holes, and stitches them with the model's probability map using thresholds tc and to, updating or ignoring each pixel. This refiner is what converts SAM's class-agnostic, boundary-sensitive masks into class-aware pseudo-labels, and it is the component whose ablation would separate the paper's contribution from ordinary self-training.

What would settle it

Take the fully trained SAMST pipeline and run the refiner with SAM's masks replaced by the thresholded connected regions themselves (i.e., no SAM re-segmentation), keeping all thresholds identical. If the mIoU gain over the baseline nearly disappears, the gain is attributable to SAM's masks; if the gain persists, the filtering/stitching alone is responsible. Alternatively, measure the pixel accuracy of refined pseudo-labels against the ground truth on the held-out portion of unlabeled data: if the refinement does not raise accuracy relative to raw predictions, the reported gains cannot come from improved label quality.

Watch

Extended reading notes

Core claim

The paper's central claim is that the SAM-based Pseudo-label Refiner, composed of a Threshold Filter Module, a Prompt Generation Module, and a Label Refinement Module, turns noisy model predictions into substantially more accurate pseudo-labels, and that this refinement is the reason SAMST improves semi-supervised segmentation. The refiner first masks out pixels whose predicted class probability falls below class-specific thresholds, then finds connected regions for each surviving class, builds prompt boxes around them, places positive and negative points inside and outside the regions, and feeds these prompts to SAM to obtain object masks. The Label Refinement Module then assigns a pixel to a class only when the SAM mask's category agrees with the model's high-confidence prediction or when the competing class is very unlikely, otherwise marking the pixel as ignored. The paper reports that after one iteration this pipeline lifts mIoU from 67.00 to 70.80 on Potsdam at a 1/32 label ratio, surpassing LSST, ST++, and ClassHyPer.

Load-bearing premise

The method's gain rests on the assumption that SAM's boundary-preserving masks, when prompted by the noisy model's boxes and points and stitched back with the probability thresholds, improve pseudo-labels instead of injecting new errors.

Editorial extensions

If this is right

  • SAM-based pseudo-label refinement yields a 5.67-point mIoU gain over the supervised baseline at a 1/32 label ratio on Potsdam.
  • A single self-training iteration suffices to surpass the three compared semi-supervised methods (LSST, ST++, ClassHyPer) on the same setup.
  • Per-class F1 rises for five of six classes, with the largest gain (17.68 points) on the clutter class; low vegetation decreases slightly.
  • Because the refiner is prompt-based and class-agnostic, the same SAMST framework can be applied to other remote sensing datasets without retraining SAM, as long as the prompt and threshold hyperparameters are re-tuned.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A controlled ablation that replaces SAM's masks with class-agnostic masks of identical coverage would reveal how much of the mIoU gain is due to SAM's boundary accuracy versus simple spatial smoothing; the paper does not report this ablation.
  • The 255 ignore-label mechanism may act as a selective curriculum: by injecting a modest number of (or zero) pseudo-labels for difficult regions, the model avoids reinforcing its own errors. This could explain the large clutter-class gain and is testable by measuring training-set class distribution before and after refinement.
  • The same refiner could be plugged into consistency-regularization methods, not just self-training, since it only touches pseudo-labels; testing SAMST's refiner inside an ST++-style pipeline would isolate whether the gains come from the refiner or the overall two-stage schedule.
  • Dataset-specific thresholds (Bn, Pp, Pn, Tp, Pm, Tn, Nm, tc, to) suggest the method may transfer less cleanly to other resolutions or class taxonomies; a sensitivity analysis over these parameters would clarify how much tuning is needed.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes SAMST, a semi-supervised semantic segmentation method for remote sensing images that combines supervised self-training with a SAM-based pseudo-label refiner. The refiner consists of a Threshold Filter Module, a Prompt Generation Module that creates boxes and positive/negative points for SAM, and a Label Refinement Module that stitches SAM masks with model probabilities. A weighted composite loss balances labeled and pseudo-labeled training. The empirical claim is that after a single iteration on the Potsdam dataset with 1/32 labeled data, SAMST improves mIoU from 67.00 (baseline) to 70.80 and outperforms LSST, ST++, and ClassHyPer in mIoU and average F1.

Significance. If fully validated, the idea of using SAM to refine pseudo-labels in semi-supervised remote sensing segmentation would be a timely and useful contribution, and the proposed pipeline is clearly motivated. The method is not circular: pseudo-labels are generated on unlabeled data and the final evaluation is on a held-out test set. However, the current evidence is not sufficient to establish the contribution: the central table has a puzzling internal inconsistency, the refiner depends on several undisclosed hand-set thresholds, no ablations are provided, and comparison settings are not described. The paper is potentially interesting but requires substantial additional detail and experiments before its claims can be accepted.

major comments (5)
  1. [Table I, Section III.C] The reported baseline OA is 93.48, which is 9.65 to 12.91 points higher than the OA of every semi-supervised comparison method (LSST 80.57, ClassHyPer 83.62, ST++ 83.83), and SAMST's OA is 86.44, a 7.04-point drop from the baseline despite higher mIoU and higher per-class F1 for Building, Tree, and Surface. Since Building, Tree, and Surface dominate the Potsdam scene, this pattern is internally inconsistent under a matched evaluation protocol. The authors should explain the experimental setup that produces this pattern, report per-class recall and confusion matrices, and verify that all methods use the same test split, evaluation code, backbone, and training budget. Without this, the reported mIoU gain cannot be attributed to SAMST.
  2. [Section II.A] The core refiner depends on seven prompt-generation parameters (Bn, Pp, Pn, Tp, Pm, Tn, Nm) and two stitching thresholds (tc, to), but the paper only states that these are 'set through experimental outcomes' and gives neither their values nor the procedure used to choose them. The same omission applies to alpha in Eq. (1) and the class weights wl,c and wp,c in Eqs. (2)-(3). Without these details the method is not reproducible, and there is no way to assess whether the choices were made on a validation set or the test set. Please report the chosen values and the validation criterion used.
  3. [Section III.B] The text says that a checkpoint is saved every 1,000 iterations and that 'the best checkpoint' is used to generate pseudo-labels, but the selection criterion is not defined and no validation split is described. If the test set was used to select among the 80 saved checkpoints, the reported numbers are optimistically biased. Please specify the validation protocol and the metric used for checkpoint selection, and apply the same protocol to the baseline and comparison methods.
  4. [Section III.C and Table I] No ablation study is provided, so it is impossible to determine which component of the proposed framework (Threshold Filter Module, Prompt Generation Module, Label Refinement Module, or the composite loss) is responsible for the reported improvement. In addition, the paper claims an iterative framework but reports only a single full iteration. Please provide module-wise ablations and at least one multi-iteration experiment to support the central claim.
  5. [Section III.B] The implementation details are given only for SAMST; there is no description of how LSST, ST++, and ClassHyPer were configured for comparison. It is therefore unclear whether the comparison methods used the same backbone, the same labeled/unlabeled split, the same input resolution, the same optimizer settings, and the same training iterations. Without matched settings, the relative gains in Table I could be an artifact of weaker baselines. Please provide full training and evaluation details for all compared methods, and ideally repeat experiments over multiple seeds to report variance.
minor comments (4)
  1. [Fig. 2] The text in panel (d) labels the class as 'Imprevious surfaces'; this should be 'Impervious surfaces'.
  2. [Section III.A] The dataset split is ambiguous: 'we allocated 24 images for training and 14 for validation and testing' is followed by a statement that the testing set includes 2016 patches. Please clarify how the validation set and test set are separated, and whether the 2016 patches come from the 14 images or from a different partition.
  3. [Section II.A.3] The Label Refinement Module description says holes are removed from each mask to obtain mask m1, but the paper does not define what 'holes' means concretely or how the masks from all connected regions are concatenated. Please provide a precise algorithmic description or pseudo-code.
  4. [References] There is no dedicated related-work section; the introduction mentions consistency regularization and self-training but does not discuss the cited remote sensing semi-supervised methods in enough detail to position the novelty of SAMST. Consider adding a short related-work section.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SAMST's headline gains rest on held-out Potsdam test-set metrics, and no fitted parameter or refined pseudo-label is renamed as the predicted target.

full rationale

Walking the paper's claimed chain shows an empirical training/evaluation pipeline rather than a derivation that reduces to its inputs. An initial model is trained on 108 labeled Potsdam patches; pseudo-labels for the remaining 3348 unlabeled patches are generated and refined by the SAM-based Pseudo-label Refiner (Threshold Filter Module, Prompt Generation Module, Label Refinement Module); the model is retrained with the composite weighted loss; and performance is then measured on the separate test portion (2016 patches). Nothing in Equations (1)-(3) defines the reported mIoU, OA, or F1 in terms of the method's own pseudo-labels or thresholds, and the refined pseudo-labels are not the evaluation target. The threshold and prompt parameters are described as set through experimental outcomes and acknowledged as dataset-specific; this is ordinary hyperparameter selection, not a fitted input renamed as a prediction. The 'best checkpoint' choice and the internal inconsistency of Table I (baseline OA 93.48 versus all semi-supervised methods below 84) are experimental-matching and reporting-integrity concerns, not circularity, because they do not make SAMST's result true by construction. There is no load-bearing self-citation and no imported uniqueness theorem. The paper is self-contained against an external benchmark, so the honest finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method is built on several hand-tuned parameters (thresholds, prompt counts, loss weight) and on strong assumptions about SAM's transferability to remote sensing. No new entities or formal derivations are introduced.

free parameters (4)
  • Threshold Filter per-class thresholds = unspecified
    Section II.A.1: predictions are filtered using pre-set class-specific thresholds; values are not reported but tuned to the dataset.
  • Prompt generation parameters (Bn, Pp, Pn, Tp, Pm, Tn, Nm) = unspecified
    Section II.A.2: 'these parameters are determined based on the dataset and categories, with specific values set through experimental outcomes.'
  • Label refinement thresholds (tc, to) = unspecified
    Section II.A.3: 'tc and to are thresholds for filtering' and their values are not provided.
  • Pseudo-loss weight alpha = unspecified
    Equation (1): Lwc = Llabel + alpha * Lpseudo; alpha is not reported.
assumptions (3)
  • domain assumption SAM's zero-shot segmentation produces accurate object boundaries for remote sensing imagery when prompted with boxes and points.
    The entire Pseudo-label Refiner relies on SAM masks being useful for land cover classes; introduced in Section II.A.
  • domain assumption Connected regions of high-confidence predictions correspond to semantically coherent objects suitable for prompting.
    The Prompt Generation Module extracts connected regions for each class and treats them as objects for SAM prompts; Section II.A.2.
  • domain assumption The threshold and prompt parameters are selected on a validation set and generalize to the test set.
    The paper does not specify a validation split and states parameters are set through experimental outcomes, so this premise is load-bearing for the reported test numbers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SAMST: A Transformer framework based on SAM pseudo label filtering for remote sensing semi-supervised semantic segmentation." pith.science (2026). https://pith.science/paper/GETPIG2J

@misc{pith2026250711994,
  author       = {Pith},
  title        = {Pith review of: SAMST: A Transformer framework based on SAM pseudo label filtering for remote sensing semi-supervised semantic segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GETPIG2J}},
  note         = {Machine review of arXiv:2507.11994}
}
read the original abstract

Public remote sensing datasets often face limitations in universality due to resolution variability and inconsistent land cover category definitions. To harness the vast pool of unlabeled remote sensing data, we propose SAMST, a semi-supervised semantic segmentation method. SAMST leverages the strengths of the Segment Anything Model (SAM) in zero-shot generalization and boundary detection. SAMST iteratively refines pseudo-labels through two main components: supervised model self-training using both labeled and pseudo-labeled data, and a SAM-based Pseudo-label Refiner. The Pseudo-label Refiner comprises three modules: a Threshold Filter Module for preprocessing, a Prompt Generation Module for extracting connected regions and generating prompts for SAM, and a Label Refinement Module for final label stitching. By integrating the generalization power of large models with the training efficiency of small models, SAMST improves pseudo-label accuracy, thereby enhancing overall model performance. Experiments on the Potsdam dataset validate the effectiveness and feasibility of SAMST, demonstrating its potential to address the challenges posed by limited labeled data in remote sensing semantic segmentation.

Figures

Figures reproduced from arXiv: 2507.11994 by the authors.

Figure 1
Figure 1. Overview of SAMST Framework with Two Stages: Supervised [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Visualization results of our proposed SAM-based Pseudo-label Refiner [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Visual Comparison of Pseudo Result with True Label. (a) depicts the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Visual Result with our proposed SAMST framework after one [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 23 canonical work pages

  1. [19]

    SAM as the Guide: Mastering Pseudo-Label Refinement in Semi-Supervised Referring Expression Segmentation

    D. Yang, J. Ji, Y . Ma, T. Guo, H. Wang, X. Sun, and R. Ji, “Sam as the guide: mastering pseudo-label refinement in semi- supervised referring expression segmentation,” arXiv preprint arXiv:2406.01451, 2024

  2. [1]

    Weak-to-strong consistency learning for semisuper- vised image segmentation,

    X. Lu, L. Jiao, L. Li, F. Liu, X. Liu, S. Yang, Z. Feng, and P. Chen, “Weak-to-strong consistency learning for semisuper- vised image segmentation,” IEEE Transactions on Geoscience and Remote Sensing , vol. 61, pp. 1–15, 2023

  3. [2]

    Classhyper: Classmix-based hybrid perturbations for deep semi-supervised semantic segmentation of remote sensing imagery,

    Y . He, J. Wang, C. Liao, B. Shan, and X. Zhou, “Classhyper: Classmix-based hybrid perturbations for deep semi-supervised semantic segmentation of remote sensing imagery,” Remote Sensing , vol. 14, no. 4, 2022. [Online]. Available: https://www.mdpi.com/2072-4292/14/4/879

  4. [3]

    Semisupervised semantic segmentation of remote sensing images with consistency self- training,

    J. Li, B. Sun, S. Li, and X. Kang, “Semisupervised semantic segmentation of remote sensing images with consistency self- training,” IEEE Transactions on Geoscience and Remote Sens- ing, vol. 60, pp. 1–11, 2022

  5. [4]

    Picoco: Pixelwise contrast and consistency learning for semisupervised building footprint segmentation,

    J. Kang, Z. Wang, R. Zhu, X. Sun, R. Fernandez-Beltran, and A. Plaza, “Picoco: Pixelwise contrast and consistency learning for semisupervised building footprint segmentation,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , vol. 14, pp. 10 548–10 559, 2021

  6. [5]

    Virtual adversarial training: A regularization method for supervised and semi-supervised learning,

    T. Miyato, S.-I. Maeda, M. Koyama, and S. Ishii, “Virtual adversarial training: A regularization method for supervised and semi-supervised learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 41, no. 8, pp. 1979– 1993, 2019

  7. [6]

    Mean teachers are better role models: Weight-averaged consistency targets improve semi- supervised deep learning results,

    A. Tarvainen and H. Valpola, “Mean teachers are better role models: Weight-averaged consistency targets improve semi- supervised deep learning results,” Advances in neural informa- tion processing systems , vol. 30, 2017

  8. [7]

    Dynamic and adaptive self-training for semi-supervised remote sensing image semantic segmentation,

    J. Jin, W. Lu, H. Yu, X. Rong, X. Sun, and Y . Wu, “Dynamic and adaptive self-training for semi-supervised remote sensing image semantic segmentation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1–14, 2024

Show all 24 references
  1. [8]

    Region-aware contrastive learning for semi-supervised semantic segmentation of remote sensing images,

    Y . Luo, B. Sun, and S. Li, “Region-aware contrastive learning for semi-supervised semantic segmentation of remote sensing images,” in IGARSS 2024 - 2024 IEEE International Geoscience and Remote Sensing Symposium , 2024, pp. 8199–8203

  2. [9]

    Semi- supervised remote sensing image semantic segmentation method based on deep learning,

    L. Li, W. Zhang, X. Zhang, M. Emam, and W. Jing, “Semi- supervised remote sensing image semantic segmentation method based on deep learning,” Electronics, vol. 12, no. 2, p. 348, 2023

  3. [10]

    Semi- supervised semantic segmentation of remote sensing images based on dual cross-entropy consistency,

    M. Cui, K. Li, Y . Li, D. Kamuhanda, and C. J. Tessone, “Semi- supervised semantic segmentation of remote sensing images based on dual cross-entropy consistency,” Entropy, vol. 25, no. 4, p. 681, 2023

  4. [11]

    St++: Make self- training work better for semi-supervised semantic segmenta- tion,

    L. Yang, W. Zhuo, L. Qi, Y . Shi, and Y . Gao, “St++: Make self- training work better for semi-supervised semantic segmenta- tion,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 4268–4277

  5. [12]

    Simple and efficient: A semisupervised learning framework for remote sensing image semantic segmentation,

    X. Lu, L. Jiao, F. Liu, S. Yang, X. Liu, Z. Feng, L. Li, and P. Chen, “Simple and efficient: A semisupervised learning framework for remote sensing image semantic segmentation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–16, 2022

  6. [13]

    Semisupervised semantic segmentation of remote sensing images with consistency self- training,

    J. Li, B. Sun, S. Li, and X. Kang, “Semisupervised semantic segmentation of remote sensing images with consistency self- training,” IEEE Transactions on Geoscience and Remote Sens- ing, vol. 60, pp. 1–11, 2021

  7. [14]

    Semi-supervised hyperspectral image classification via spatial- regulated self-training,

    Y . Wu, G. Mu, C. Qin, Q. Miao, W. Ma, and X. Zhang, “Semi-supervised hyperspectral image classification via spatial- regulated self-training,” Remote Sensing, vol. 12, no. 1, p. 159, 2020

  8. [15]

    Semi-supervised semantic segmentation with cross-consistency training,

    Y . Ouali, C. Hudelot, and M. Tami, “Semi-supervised semantic segmentation with cross-consistency training,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 12 674–12 684

  9. [16]

    Semi-supervised semantic segmentation with high-and low-level consistency,

    S. Mittal, M. Tatarchenko, and T. Brox, “Semi-supervised semantic segmentation with high-and low-level consistency,” IEEE transactions on pattern analysis and machine intelligence, vol. 43, no. 4, pp. 1369–1379, 2019

  10. [17]

    Pseudo-label: The simple and efficient semi- supervised learning method for deep neural networks,

    D.-H. Lee et al., “Pseudo-label: The simple and efficient semi- supervised learning method for deep neural networks,” in Work- shop on challenges in representation learning, ICML , vol. 3, no. 2. Atlanta, 2013, p. 896

  11. [18]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo et al. , “Segment anything,” in Proceedings of the IEEE/CVF international conference on computer vision , 2023, pp. 4015– 4026

  12. [20]

    The isprs benchmark on urban object classification and 3d building reconstruction,

    F. Rottensteiner, G. Sohn, J. Jung, M. Gerke, C. Baillard, S. Benitez, and U. Breitkopf, “The isprs benchmark on urban object classification and 3d building reconstruction,” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences; I-3 , vol. 1, no. ...

  13. [21]

    What is a good evaluation measure for semantic segmentation?

    G. Csurka, D. Larlus, F. Perronnin, and F. Meylan, “What is a good evaluation measure for semantic segmentation?.” in Bmvc, vol. 27, no. 2013. Bristol, 2013, pp. 10–5244

  14. [22]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF in- ternational conference on computer vision , 2021, pp. 10 012– 10 022

  15. [23]

    Unified perceptual parsing for scene understanding,

    T. Xiao, Y . Liu, B. Zhou, Y . Jiang, and J. Sun, “Unified perceptual parsing for scene understanding,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 418–434

  16. [24]

    Context encoding for semantic segmentation,

    H. Zhang, K. Dana, J. Shi, Z. Zhang, X. Wang, A. Tyagi, and A. Agrawal, “Context encoding for semantic segmentation,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2018, pp. 7151–7160

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.