Pith. sign in

REVIEW 3 major objections 5 minor 16 references

PBVS 2024 Solution: Self-Supervised Learning and Sampling Strategies for SAR Classification in Extreme Long-Tail Distribution

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read In an extreme long-tail SAR classification challenge, a two-stage self-supervised pipeline with undersampled KNN ensembles reached 21.45% accuracy and 9th place.

desk verdict Honest competition write-up with a real external score, but the paper never shows the pipeline caused that score. read the letter →

arxiv 2412.12565 v1 pith:4M5LCZOO submitted 2024-12-17 cs.CV

classification cs.CV
keywords SARclassificationlong-taillearningself-supervisedDINOv2SAR-to-EOtranslationLeefilterundersamplingKNNensemble
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper confronts a vehicle-classification task in which the largest class has 364,291 training samples and the smallest has 353, an imbalance of more than 1000 to 1 that defeats standard long-tail remedies. It argues that the workable recipe is a two-stage design: first train a self-supervised vision transformer (DINOv2) on the full unlabeled set so that rare-class structure is not sacrificed, then balance the feature space with Tomek Links and NearMiss-3 and combine the predictions of several K-nearest-neighbor classifiers, each trained on a balanced subset. To bridge the radar-optical domain gap, each input is a three-channel image made of the original SAR view, a Lee-filtered (despeckled) SAR view, and a SAR-to-EO translated view. On the held-out test set this system posted 21.45% accuracy, 0.56 AUC, and a total score of 0.30, ranking 9th in the challenge. The paper's contribution is the specific combination of these ingredients for an extreme long-tail setting, rather than a claim that any individual stage is sufficient.

What carries the argument

The central mechanism is a two-stage pipeline built around a frozen self-supervised vision transformer (DINOv2). Stage one learns representations on the entire dataset without labels so that head-class bias does not enter the feature extractor; stage two uses those features to construct balanced training subsets via Tomek Links and NearMiss-3 and trains an ensemble of distance-based KNN classifiers (K=3) on them. The input representation is equally load-bearing: a three-channel image obtained by concatenating the original SAR, a Lee-filtered (despeckled) SAR, and a synthetic EO image produced by Pix2PixHD, so the network can exploit complementary views without explicit multimodal fusion. Each piece is designed to counter one of the two stated failure modes, the extreme class imbalance and the SAR/EO domain disparity.

What would settle it

Run the same DINOv2-KNN ensemble on single-channel original SAR images, then add the Lee-filtered channel and then the translated channel, and compare total scores on the PBVS 2024 validation or test set; if the score does not drop when the extra channels are removed, the claim that the multimodal preprocessing helps is falsified. In parallel, compute per-class accuracy for the smallest classes with and without Tomek Links and NearMiss-3; absence of improvement there would falsify the balancing claim.

Watch

Extended reading notes

Core claim

The central claim is that a severe long-tail SAR classification problem can be handled without ever training a supervised classifier on raw imbalanced data, and that the radar-optical domain gap can be folded into the input representation rather than into a custom fusion network. DINOv2 is first trained self-supervised on all 455,600 samples; then, in the frozen feature space, Tomek Links removes majority samples that sit close to minority samples and NearMiss-3 selects the most ambiguous remaining samples, producing several balanced subsets. A KNN classifier with K=3 is trained on each subset and the outputs are ensembled. In parallel, every SAR image is despeckled with a Lee filter and translated into an EO-style image with Pix2PixHD, and the original, filtered, and translated views are concatenated into a three-channel input used at both training and inference. The authors report that this pipeline achieved 21.45% accuracy, 0.56 AUC, and a total score of 0.30, placing 9th in the PBVS 2024 MAVIC-C.

Load-bearing premise

The paper assumes each pipeline component helps, because it reports no experiment that removes any single component to check whether the gains actually come from the combination.

Editorial extensions

If this is right

  • If the pipeline works as described, the self-supervised feature extractor can be trained on the full unlabeled dataset before any balancing, so rare-class information is not discarded during pretraining.
  • The three-channel input, composed of original, Lee-filtered, and SAR-to-EO translated views, is used at both training and inference, giving the model access to both modalities without a separate fusion network.
  • The combination of Tomek Links, NearMiss-3, and ensembled KNN classifiers is the paper's proposed way to build a balanced classifier in an extreme 1000-to-1 long-tail setting.
  • The method's final numbers on the PBVS 2024 test set are 21.45% accuracy, 0.56 AUC, and a total score of 0.30, ranking 9th.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The paper does not report per-class accuracy, but its own logic predicts that the undersampling and ensembling steps earn their keep on the smallest classes; a reader could test this by computing rare-class accuracy with and without Tomek Links and NearMiss-3.
  • Inference: Since no ablation is given, the natural follow-up experiment is to run the same DINOv2-KNN ensemble on the original SAR channel alone and then add the Lee-filtered and translated channels one at a time; if the total score does not move, the multimodal preprocessing is not the active ingredient.
  • Inference: The dependence on Pix2PixHD for SAR-to-EO translation adds a generation step at inference time, which could be a practical bottleneck; replacing it with lighter synthesis or direct feature-space alignment is an obvious extension of the approach.
  • Inference: The same two-stage recipe could be transported to other paired-sensor tasks (for instance optical-to-infrared) where one view is noisy and class counts differ by orders of magnitude, though the paper itself does not claim this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper describes the authors' solution for the PBVS 2024 Multi-modal Aerial View Imagery Challenge Classification (MAVIC-C) SAR classification task. The proposed pipeline consists of (i) preprocessing SAR inputs with a Lee filter and Pix2PixHD-based SAR-to-EO translation, then concatenating original SAR, denoised SAR, and synthetic translated EO into a three-channel input; (ii) self-supervised representation learning with DINOv2; (iii) long-tail handling by Tomek Links and NearMiss-3 undersampling of DINOv2 features to create seven balanced subsets; and (iv) ensembling K-nearest-neighbor classifiers trained on each subset. The authors report official test-set results of 21.45% accuracy, 0.56 AUC, and a total score of 0.30, placing 9th in the competition, and attribute this outcome to their proposed components.

Significance. If the claimed benefits were supported by evidence, the combination of SAR denoising, SAR-to-EO translation, and instance-based undersampling with self-supervised DINOv2 features would be of practical interest for extreme long-tail SAR classification. The use of an external competition test set is a genuine strength: the reported score avoids training/evaluation circularity. However, the manuscript contains no baseline, no ablation, and no sensitivity analysis, so the contribution of any individual component is unverified. The reported result is also modest—9th place, with total score 0.30 compared to 0.49 for the winning team—so the practical significance of the pipeline is not demonstrated as submitted.

major comments (3)
  1. [Sections 2 and 3] The central empirical claim of the paper—that Lee filtering, SAR-to-EO translation, three-channel concatenation, Tomek Links, NearMiss-3, and ensembling improve long-tail SAR classification—is unsupported. The only quantitative result is the official competition score reported in Section 3 and Table 1; no comparison is made against a plain DINOv2-backed KNN baseline, single-channel or two-channel inputs, no-resampling training, or a single KNN classifier. Without such ablations, the final score could be produced by the pretrained DINOv2 feature extractor and KNN alone, and the causal statements in Section 4 ('these balanced subsets provided an optimal foundation', 'the ensemble further boosts robustness') are unverifiable. Please add ablations on a validation split, or on the frozen challenge test set if additional submissions are permitted, isolating each proposed component.
  2. [Section 1 vs. Section 2.2] The Introduction promises a clustering component ('we combine clustering and self-supervised learning', 'Clustering helps identify representative samples'), but no clustering algorithm appears in Sections 2.2 or 3. Tomek Links and NearMiss-3 are undersampling and instance-selection techniques, not clustering, and KNN is a classifier, not a clustering procedure. This internal inconsistency must be resolved: either add a clustering step to the pipeline or revise the Introduction to describe the actual undersampling procedure.
  3. [Section 2.1] The claim that concatenating original SAR, Lee-filtered SAR, and SAR-to-EO translated images into a three-channel input 'mitigates' the shape distortion introduced by denoising and translation is asserted without experiment. Moreover, although the challenge provides real EO images for training, the paper never describes using them in classification; only synthetic EO from SAR-to-EO translation appears in the input. The role of each modality and the benefit of multi-channel concatenation should be clarified and empirically tested.
minor comments (5)
  1. [Figure 3] The caption should identify which panel is which (original SAR, Lee-filtered SAR, translated EO, reference EO); the current caption '(d)(c)(b)(a)' is confusing.
  2. [Section 3] The paper says DINOv2 was 'trained' on the entire dataset but also that it was 'initialized with pre-trained weights from DINOv2'; specify whether this is continued self-supervised pretraining or fine-tuning, and report training epochs, loss, and compute.
  3. [Section 2.2] The construction of the 'N balanced subsets' is not described; explain how Tomek Links and NearMiss-3 are applied to high-dimensional DINOv2 features and how the seven subsets are generated (e.g., bootstrap or disjoint partitions).
  4. [Table 1] The total score formula used to rank teams is not defined; without it, the relation between accuracy, AUC, and the final ranking is unclear.
  5. [Throughout] The text alternates between 'DINOv2' and 'Dinov2'; please use consistent capitalization.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported scores are measured on an external PBVS 2024 test set, and no claimed result reduces to a fitted input or a load-bearing self-citation.

full rationale

The paper makes no predictive claim that reduces to a fitted constant. Its headline outcome (accuracy 21.45%, AUC 0.56, total score 0.30, 9th place) is measured on the PBVS 2024 competition test set, which was not used to fit the model, so the training-evaluation loop is externally grounded and not circular. The preprocessing and sampling components (Lee filter, Pix2PixHD translation, three-channel concatenation, Tomek Links, NearMiss-3, and KNN ensembling) are introduced as design choices, not as quantities fitted to the test outcome; hyperparameters such as K=3 and 7 subsets are hand-set in Section 3. The self-citations ([4], [5]) are background references for SAR-to-EO translation and multimodal distillation and are not load-bearing, since the actual translation uses the external Pix2PixHD model [12] and the feature extractor is the external DINOv2 model [7]. The absence of ablations or baselines, and the mismatch between the Introduction's promised 'clustering' and the Method's actual sampling procedures, are correctness and evidential weaknesses, but they are not circular dependencies: no component is defined in terms of its own output, and no fitted parameter is renamed as a prediction. Therefore, no circular step meeting the stated evidentiary standard is present.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper relies on several untested domain assumptions about the usefulness of its components and the validity of the single competition evaluation. There are no invented entities. Free parameters are limited to hand-chosen hyperparameters and unspecified preprocessing details.

free parameters (5)
  • KNN number of neighbors K = 3
    Set to 3 by hand; no sensitivity analysis provided.
  • Number of ensemble subsets N = 7
    Ensemble comprised 7 balanced subsets; choice not justified by experiments.
  • Input resize size = 56x56
    Images resized to 56x56 pixels; no ablation on this choice.
  • Lee filter parameters = not specified
    Lee filter applied but its window size and parameters are not reported.
  • Pix2PixHD training hyperparameters = not specified
    Details of the SAR-to-EO translation model training are omitted.
assumptions (5)
  • domain assumption The PBVS 2024 test set is a fair and unbiased evaluation of model performance.
    The paper uses a single competition test run; no repeated trials or confidence intervals, and it assumes the competition evaluation is correct.
  • ad hoc to paper SAR-to-EO translation with Pix2PixHD produces images whose complementary information improves classification.
    No ablation compares 3-channel input against using only SAR channels; this is an unverified modeling assumption in Section 2.1.
  • domain assumption Tomek Links and NearMiss-3 undersampling improve minority-class accuracy in extreme long-tail settings.
    The paper assumes these sampling techniques are beneficial based on prior literature, but no experiments isolate their effect. See Section 2.2.
  • domain assumption DINOv2 self-supervised features transfer to SAR imagery.
    The model is initialized with DINOv2 pretrained on natural images; the paper assumes these features are useful for SAR without ablation.
  • ad hoc to paper Concatenating original SAR, denoised SAR, and translated EO images as three channels yields complementary information.
    This is a key modeling choice (Section 2.1) with no supporting ablation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PBVS 2024 Solution: Self-Supervised Learning and Sampling Strategies for SAR Classification in Extreme Long-Tail Distribution." pith.science (2026). https://pith.science/paper/4M5LCZOO

@misc{pith2026241212565,
  author       = {Pith},
  title        = {Pith review of: PBVS 2024 Solution: Self-Supervised Learning and Sampling Strategies for SAR Classification in Extreme Long-Tail Distribution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4M5LCZOO}},
  note         = {Machine review of arXiv:2412.12565}
}
read the original abstract

The Multimodal Learning Workshop (PBVS 2024) aims to improve the performance of automatic target recognition (ATR) systems by leveraging both Synthetic Aperture Radar (SAR) data, which is difficult to interpret but remains unaffected by weather conditions and visible light, and Electro-Optical (EO) data for simultaneous learning. The subtask, known as the Multi-modal Aerial View Imagery Challenge - Classification, focuses on predicting the class label of a low-resolution aerial image based on a set of SAR-EO image pairs and their respective class labels. The provided dataset consists of SAR-EO pairs, characterized by a severe long-tail distribution with over a 1000-fold difference between the largest and smallest classes, making typical long-tail methods difficult to apply. Additionally, the domain disparity between the SAR and EO datasets complicates the effectiveness of standard multimodal methods. To address these significant challenges, we propose a two-stage learning approach that utilizes self-supervised techniques, combined with multimodal learning and inference through SAR-to-EO translation for effective EO utilization. In the final testing phase of the PBVS 2024 Multi-modal Aerial View Image Challenge - Classification (SAR Classification) task, our model achieved an accuracy of 21.45%, an AUC of 0.56, and a total score of 0.30, placing us 9th in the competition.

Figures

Figures reproduced from arXiv: 2412.12565 by the authors.

Figure 1
Figure 1. Provided SAR-EO Pair Dataset. View Imagery Challenge - Classification, a sub-challenge of the PBVS 2024 workshop [6], participants are tasked with identifying ten different types of vehicles using Syn￾thetic Aperture Radar (SAR) data, which is robust against weather and visible light conditions. Electro-Optical (EO) image data is also provided but only for training purposes. The dataset features a severe long-tail d… view at source ↗
Figure 2
Figure 2. Overview of Our Proposed Pipeline ture extractor is biased towards head classes, influencing the second-stage classifier to overfit the head class. Further￾more, uniform sampling often leaves a large portion of the data underutilized, reducing generalization performance. Secondly, the domain disparity between SAR and EO images. Traditional multimodal methods [3, 5, 14, 16] pre￾dominantly rely on knowledge distillati… view at source ↗
Figure 3
Figure 3. Types of Data Used in Training. (a) is the original SAR image. (b) The SAR image with a Lee filter applied. (c) The result of translating the SAR image to EO using the Pix2PixHD model, and (d) The original EO image for comparison with (c). preserving the structural details of the images, enhancing classifier performance. To further improve accuracy and robustness by lever￾aging the complementary information between … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 13 canonical work pages

  1. [1]

    Sar-to-eo image translation with multi- conditional adversarial networks

    Armando Cabrera, Miriam Cha, Prafull Sharma, and Michael Newey. Sar-to-eo image translation with multi- conditional adversarial networks. In 2021 55th Asilomar Conference on Signals, Systems, and Computers , pages 1710–1714. IEEE, 2021. 2

  2. [2]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 3

  3. [3]

    Distill- ing the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distill- ing the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. 2

  4. [4]

    Clean collector algorithm for satel- lite image pre-processing of sar-to-eo translation

    Min-Woo Kim, Se-Kil Park, Jin-Gi Ju, Hyeon-Cheol Noh, and Dong-Geol Choi. Clean collector algorithm for satel- lite image pre-processing of sar-to-eo translation. Electron- ics (2079-9292), 13(22), 2024. 2

  5. [5]

    Privacy-safe action recognition via cross-modality distillation

    Yuhyun Kim, Jinwook Jung, Hyeoncheol Noh, Byungtae Ahn, JungHye Kwon, and Dong-Geol Choi. Privacy-safe action recognition via cross-modality distillation. IEEE Ac- cess, 2024. 2

  6. [6]

    Multi-modal aerial view image challenge: Sar classification

    Spencer Low, Oliver Nina, Dylan Bowald, Angel D Sappa, Nathan Inkawhich, and Peter Bruns. Multi-modal aerial view image challenge: Sar classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3105–3112, 2024. 1

  7. [7]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth’ee Darcet, Th’eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2, 3

  8. [8]

    Automatic target recognition (atr) from sar imaginary by using machine learning techniques

    Umut ¨Ozkaya. Automatic target recognition (atr) from sar imaginary by using machine learning techniques. Avrupa Bilim ve Teknoloji Dergisi, pages 165–169, 2020. 1

Show all 16 references
  1. [9]

    Deep mul- timodal learning: A survey on recent advances and trends

    Dhanesh Ramachandram and Graham W Taylor. Deep mul- timodal learning: A survey on recent advances and trends. IEEE signal processing magazine, 34(6):96–108, 2017. 1

  2. [10]

    Distributional robustness loss for long-tail learning

    Dvir Samuel and Gal Chechik. Distributional robustness loss for long-tail learning. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 9495–9504,

  3. [11]

    Improved flood insights: Diffusion-based sar to eo image translation

    Minseok Seo, Youngtack Oh, Doyi Kim, Dongmin Kang, and Yeji Choi. Improved flood insights: Diffusion-based sar to eo image translation. arXiv preprint arXiv:2307.07123 ,

  4. [12]

    High-resolution image syn- thesis and semantic manipulation with conditional gans

    Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High-resolution image syn- thesis and semantic manipulation with conditional gans. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8798–8807, 2018. 2, 3

  5. [13]

    Sar image despeckling using refined lee filter

    Aiyeola Sikiru Yommy, Rongke Liu, and Shuang Wu. Sar image despeckling using refined lee filter. In 2015 7th Inter- national Conference on Intelligent Human-Machine Systems and Cybernetics, volume 2, pages 260–265. IEEE, 2015. 2

  6. [14]

    Cmx: Cross-modal fusion for rgb-x semantic segmentation with transformers

    Jiaming Zhang, Huayao Liu, Kailun Yang, Xinxin Hu, Ruip- ing Liu, and Rainer Stiefelhagen. Cmx: Cross-modal fusion for rgb-x semantic segmentation with transformers. IEEE Transactions on Intelligent Transportation Systems, 2023. 2

  7. [15]

    Distribution alignment: A unified frame- work for long-tail visual recognition

    Songyang Zhang, Zeming Li, Shipeng Yan, Xuming He, and Jian Sun. Distribution alignment: A unified frame- work for long-tail visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2361–2370, 2021. 1

  8. [16]

    Cmot: Cross- modal mixup via optimal transport for speech translation

    Yan Zhou, Qingkai Fang, and Yang Feng. Cmot: Cross- modal mixup via optimal transport for speech translation. arXiv preprint arXiv:2305.14635, 2023. 2

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.