Pith. sign in

REVIEW 5 major objections 5 minor 60 references

RRS-10K: A Multitask Vision-Language Model Benchmark for Rare Remote Sensing Image Interpretation

T0 review · 5 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current vision-language models are only moderately capable on rare remote sensing scenes, and RRS-10K is designed to measure exactly where they fail.

desk verdict RRS-10K is a needed rare-scene benchmark, but its visual-grounding column looks like a parsing artifact — don't cite the VG numbers until the parser is released. read the letter →

arxiv 2607.24810 v1 pith:KE7CA2Z7 submitted 2026-07-13 cs.AI eess.IV

Yuqiao Lai , Jiancheng Qi , Fei Wang , Yuxin Liu , Kun Li , Ye Chen , Yan Gao , Yanyan Wei This is my paper
classification cs.AIeess.IV
keywords rareremotesensingimageinterpretationvision-languagemodelsbenchmarkvisualgroundingreferringsegmentationsemanticreasoningrobustnessmilitaryscenes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that today's vision-language models, which look strong on common urban and rural remote sensing imagery, are only moderately capable on rare scenes such as military installations, and that the gap is concentrated in visual grounding, referring segmentation, and complex semantic reasoning. To make this claim testable, the authors built RRS-10K, a benchmark of 10,738 first-hand images organized into three capability dimensions, six sub-dimensions, and twenty leaf tasks, covering perception, reasoning, and robustness. They also introduce a distractor-filtering strategy to keep multiple-choice questions neither too easy nor too ambiguous, and they evaluate 52 models. If the benchmark is valid, it provides a diagnostic decomposition of rare-scene capability gaps rather than just a ranking.

What carries the argument

The central object is RRS-10K itself: a hierarchical benchmark built from 10,738 rare-scene remote sensing images with multiple answer formats, organized into perception, reasoning, and robustness dimensions that decompose into 20 leaf tasks. The construction machinery that carries the argument is two-part: a human-centered hybrid annotation pipeline that combines expert descriptions, open-source references, and model-generated candidates later corrected by humans; and the similarity-based distractor filtering strategy (SDFS), which ranks a predefined option pool with a contrastive image-text encoder and keeps only distractors in a moderate-similarity band, so that multiple-choice questions

What would settle it

Re-score the visual grounding subset after normalizing model outputs into several accepted coordinate formats (bracketed, unbracketed, decimal vs. percent, JSON vs. plain text) and compare the resulting Acc@0.5 scores. If open-source models jump from near zero to the level of the best proprietary model, the claimed grounding deficit is a scoring artifact. A complementary check is to audit a random sample of the ground-truth boxes and masks produced by the automatic detection and segmentation tools, since annotation errors would be inherited by every model score.

Watch

Extended reading notes

Core claim

The central claim is that a systematic evaluation of 52 models on rare remote sensing imagery reveals a reproducible capability hierarchy: coarse scene understanding and image description are comparatively mature, while visual grounding, referring segmentation, distance and area reasoning, and robustness to noise or occlusion remain major bottlenecks. The paper argues that this pattern reflects weak coupling between global scene semantics and fine-grained local evidence, and that leading open-source models are now broadly competitive with proprietary ones, with model scale alone insufficient to explain performance. RRS-10K is presented as the instrument that makes these bottlenecks visible a

Load-bearing premise

The load-bearing premise is that the evaluation harness correctly parses and scores visual-grounding and segmentation outputs; in the task-level results table, the Visual Grounding column collapses to 0.00 for nearly all open-source models while a single proprietary model reaches about 50, and if those zeros are a format or parsing artifact, the paper's headline 'clear weakness' claim is substantially weakened.

Editorial extensions

If this is right

  • If RRS-10K measures what it claims, then even the strongest evaluated models, averaging around 71 percent overall, are far from reliable on rare-scene interpretation, with visual grounding and referring segmentation near zero for most models.
  • The 20-task taxonomy turns the benchmark into a diagnostic tool: future work can target specific bottlenecks, such as the coupling of global scene semantics with local evidence, instead of optimizing a single aggregate score.
  • The reported competitiveness of leading open-source models implies that rare-scene remote sensing research can proceed with open baselines rather than depending on proprietary APIs.
  • The robustness results imply that hallucination suppression is largely solved relative to noise and occlusion robustness, so reliability work should concentrate on degraded and partially occluded imagery.
  • Because the benchmark uses first-hand imagery and a difficulty-aware distractor filter, score gaps are more likely to reflect genuine capability differences than leakage from pretraining corpora or trivially easy options.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The near-zero visual grounding scores for almost all open-source models, against a single proprietary score near 50 percent, look like a coordinate-format or parsing artifact; re-scoring with flexible output parsing could change the reported grounding ranking.
  • Because the same model used to draft annotations and referring expressions is also among the evaluated systems, the benchmark may implicitly favor that model's phrasing; a human-generated control set would test this.
  • The hierarchical taxonomy could be ported to other long-tail domains — disaster response, marine surveillance, or critical infrastructure — by replacing the scene categories while keeping the capability dimensions.
  • The paper evaluates zero-shot only; a natural next test is whether fine-tuning on RRS-10K subtasks closes the identified bottlenecks or merely overfits the benchmark.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. RRS-10K introduces a benchmark of 10,738 military-related remote sensing images for evaluating vision-language models (VLMs) on rare-scene interpretation. The benchmark is organized into 3 capability dimensions, 6 sub-dimensions, and 20 leaf tasks spanning perception, reasoning, and robustness, using multiple answer formats (MCQ, open-ended, bounding boxes, masks). Construction combines expert knowledge, open-source textual augmentation, GPT-5.4-assisted annotation, a CLIP-based similarity distractor filtering strategy (SDFS), and human review. The authors evaluate 43 VLMs and 9 referring-segmentation models and report that current models perform only moderately, with clear weaknesses in visual grounding, referring segmentation, and complex semantic reasoning, while leading open-source models are becoming competitive with proprietary systems.

Significance. RRS-10K addresses a real gap: rare and military remote sensing scenes are underrepresented in current VLM benchmarks. The paper's strengths are the scale (10,738 images, 24 countries), the hierarchical 20-task taxonomy, the 52-model evaluation, the SDFS distractor-filtering idea, and the multi-format annotation pipeline. If the measurement issues are resolved, the benchmark would be a useful diagnostic resource, and the finding that open-source models are competitive on many tasks would be informative. At present, however, several load-bearing evaluation choices are not validated, so the headline claims are not yet established at the level the paper asserts.

major comments (5)
  1. [§IV-B(3), Eq. (9); §III-B.2.4; Fig. 6; Table II] The VG column in Table II is bimodal: GPT-5.4 scores 49.89, Claude Sonnet 4.6 25.80, GPT-4o 14.93, while nearly all open-source models score 0.00–0.86 and Gemini-3-Flash scores 0.00 despite leading most other tasks. The manuscript defines Acc@τ but never states which τ is used for Table II and, more importantly, never specifies how raw model outputs are parsed and normalized into (x1,y1,x2,y2) coordinates. A strict regex parser expecting one exact layout would produce exactly the observed pattern. The prompt example in Fig. 6 requests the format '(x1,y1, x2,y2)', but no evidence is provided that all evaluated models' output formats are handled. Because the abstract and Section V identify visual grounding as a clear weakness, this is a load-bearing measurement question. Please release the parser, report unparseable rates, re-run the evaluations, or otherwise validate that the zeros reflec
  2. [§III-B.2.3, §III-B.2.4, §III-B.3; Table II] GPT-5.4 generated the preliminary question-answer-reasoning annotations, the referring expressions for VG and segmentation, and the hallucination-detection questions, and GPT-5.4 is then evaluated on this same benchmark. The manuscript asserts human review and correction but provides no quantitative edit rate, inter-annotator agreement, or analysis of residual model bias. The hallucination task is especially circular because GPT-5.4 wrote the deceptive prompts it is later tested against. This is not necessarily intentional bias, but it prevents the GPT-5.4 scores from being interpreted as an independent measurement. Please provide a human-authored held-out subset, exclude GPT-5.4 from headline rankings, or report per-item provenance and revision statistics.
  3. [§IV-C, Tables II–IV, Fig. 9] All comparisons are point estimates; there are no confidence intervals, bootstrap replicates, or significance tests across the 20 tasks or 52 models. Several headline differences are small (e.g., Qwen3-VL-235B-Instruct 69.63 vs GPT-4o 69.60 in Table II; Qwen3-VL-4B 62.82 vs Claude Sonnet 4.6 62.58). The claims that proprietary and open-source models are 'broadly competitive' and that scaling is 'not uniformly decisive' depend on rankings that may be within sampling noise. Add uncertainty quantification for the aggregated and task-level scores.
  4. [§III-B.3.2] SDFS is described only qualitatively: distractors are selected from the 'middle portion' of a CLIP similarity ranking, but no cutoffs, sampling rule, or number of candidates are specified. This makes the MCQ construction non-reproducible and leaves open the possibility that the benchmark's difficulty is coupled to CLIP's similarity space. Please release the similarity thresholds, per-item scores, and the list of rejected/kept options, and report human revision statistics. Without this, the MCQ Acc values in Tables II–VI cannot be independently reconstructed.
  5. [§III-C.2 and Fig. 7] Distance Calculation and Area Calculation require converting pixel coordinates to physical meters, but the manuscript does not describe how the metric-scale ground-truth labels were derived, which images contain usable scale cues, or how the 'weak-supervision metric-scale information' was extracted. The high DC/AC scores (e.g., Qwen3-VL-235B-Instruct 85.41 and 70.00 in Table II) are not interpretable without this protocol. Specify the conversion rule and exclude or flag samples without reliable scale information.
minor comments (5)
  1. [§III-C.1 and Fig. 6] Typographical errors: 'PA VEPAW radar' appears in the scene-classification description and the VG example in Fig. 6 contains '0.635.0.213' instead of '0.635,0.213'.
  2. [§III-B.1 vs. Abstract] The abstract states images are 'collected from first-hand sources,' but Section III-B.1 says data come from 'publicly available sources and commercial satellite imagery providers, including Google Earth, Maxar, Airbus, and Planet Labs.' Please clarify what 'first-hand' means and specify data licenses and redistribution terms.
  3. [§IV-B(2)] Image Description is scored with BERTScore-F1, and the reported scores cluster tightly around 88–91 even for OCR-specialized models that fail badly on other tasks. BERTScore-F1 appears to saturate for these generic descriptions. Consider adding a more discriminative metric (e.g., CIDEr or human ratings) or reporting breakdowns by semantic content.
  4. [§III-B.1] The paper criticizes existing benchmarks for possible pretraining overlap, but it does not provide a leakage or near-duplicate analysis for RRS-10K images sourced from Google Earth, Maxar, Airbus, and Planet Labs. Please add a decontamination check or at least discuss the residual risk of overlap with VLM training corpora.
  5. [Table III] The column label 'Geomatic & Spatial Inference' should be 'Geometric-Spatial Inference' for consistency with Section III-C.2.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found; the reported claims are empirical measurements, with a benchmark-validity caveat about GPT-5.4-assisted annotation and VG parsing.

full rationale

RRS-10K is a benchmark/evaluation paper rather than a predictive derivation. Equations (1)-(11) define standard metrics (accuracy, BLEU/ROUGE/BERTScore, IoU/Acc@tau, mIoU/Dice); none of these take the construction pipeline as an input, so no computed score is equivalent to a fitted parameter. The main results (Tables II-VI) are measurements of 52 external models against human-reviewed ground truth. SDFS is explicitly anti-circular: CLIP similarities rank/filter distractors "rather than for final label prediction," and the GT is "always inherited from the original ground-truth annotation." The use of GPT-5.4 to draft questions/referring expressions and of Grounding DINO/SAM-3 to propose boxes/masks, with later human correction (Section III-B.2.3-2.4), creates a potential leakage/contamination risk for GPT-5.4's scores, but the paper does not define any score as, or derive any claim from, GPT-5.4's own outputs; the reported task accuracies are not the annotations themselves. Likewise, the bimodal VG scores and the absence of a stated output-parsing protocol in Section IV-B are measurement-validity concerns, not a circular reduction: no equation or rule equates non-conforming text to zero. The paper asserts "All generated annotations are subsequently reviewed and corrected by human annotators" without quantifying this review, and the VG parser is not specified; these are transparency/support gaps, not circularity. The only self-citation is [7], a general object-detection survey in related work, which is not load-bearing. Under the stated standard, no step reduces the paper's conclusions to its inputs by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The benchmark's quantitative output rests on label quality, legal redistributability of commercial imagery, absence of pretraining leakage, and an evaluation harness that parses all model formats. None of these is demonstrated; the first and fourth are the most load-bearing.

free parameters (4)
  • SDFS similarity-range cutoffs = not specified
    Section III-B.3.2 discards candidates at both similarity extremes and keeps the 'middle portion'; the numeric cutoffs are never given, so distractor difficulty selection is not reproducible.
  • IoU threshold tau in Acc@tau = not specified
    Section IV-B Eq. (9) defines grounding accuracy as Acc@tau with 'standard grounding protocols [53]' but never states the tau value; VG results depend directly on it.
  • Noise and occlusion perturbation parameters = not specified
    Section III-B.3 describes Gaussian noise, salt-and-pepper noise, and 'random occlusion' with no severity, fraction, or seed; NR/OR scores are therefore not reproducible.
  • Metric-scale conversion for distance/area labels = not specified
    Section III-B.2 and Fig. 5 state that distance/area GT come from 'weak-supervision cues inherent in original imagery, including directional and metric-scale information'; the pixel-to-meter mapping is never validated, yet DC/AC accuracy scores (Table VI) treat it as ground truth.
assumptions (4)
  • domain assumption Model-generated annotations (GPT-5.4 QAs, Grounding DINO boxes, SAM-3 masks) are accurate enough after human review to serve as ground truth.
    Section III-B.2.3-2.4: VG and segmentation GT are produced by these tools and then 'verified and corrected by human annotators.' If the review was not exhaustive, label noise propagates into every accuracy number.
  • domain assumption Imagery from Google Earth, Maxar, Airbus, and Planet Labs may be legally redistributed as a public benchmark.
    Section III-B.1 names these commercial providers but gives no license or terms-of-service analysis. Redistribution rights are essential for the benchmark to exist as a shared resource.
  • domain assumption The 10,738 images do not overlap with the pretraining corpora of the 52 evaluated VLMs.
    Section II-B criticizes prior benchmarks for leakage from captioning/VQA datasets, but no deduplication or membership test is reported for RRS-10K images against GPT-4o/Qwen/Gemini training data, despite the images being sourced from the open web.
  • ad hoc to paper GPT-5.4-generated questions and reasoning, after human editing, are unbiased for evaluating GPT-5.4 itself.
    Section III-B.2.3: GPT-5.4 generates preliminary annotations and reasoning; Table II then evaluates GPT-5.4 on those very annotations. The claimed human verification has no reported statistics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RRS-10K: A Multitask Vision-Language Model Benchmark for Rare Remote Sensing Image Interpretation." pith.science (2026). https://pith.science/paper/KE7CA2Z7

@misc{pith2026260724810,
  author       = {Pith},
  title        = {Pith review of: RRS-10K: A Multitask Vision-Language Model Benchmark for Rare Remote Sensing Image Interpretation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KE7CA2Z7}},
  note         = {Machine review of arXiv:2607.24810}
}
read the original abstract

Vision-language models (VLMs) have achieved strong performance on general remote sensing tasks. However, their capability for rare scenes remains insufficiently understood, because existing benchmarks are dominated by common urban and rural imagery. To address this gap, we present RRS-10K, a benchmark for rare remote sensing image interpretation. RRS-10K contains 10,738 military-related remote sensing images and corresponding multiple format question-answer pairs for comprehensive evaluation. All of the images are collected from first-hand sources and organized into three capability dimensions, six sub-dimensions, and 20 leaf tasks, covering perception, reasoning, and robustness. To improve the quality of multiple-choice questions, we introduce a similarity-based distractor filtering strategy (SDFS) during benchmark construction. We further evaluate 52 representative models and show that current VLMs achieve only moderate zero-shot performance on rare remote sensing image interpretation, with clear weaknesses in visual grounding, referring segmentation, and complex semantic reasoning tasks. RRS-10K enables systematic analysis of failure modes in long-tail remote sensing interpretation and provides guidance for developing more reliable remote sensing VLMs.

Figures

Figures reproduced from arXiv: 2607.24810 by the authors.

Figure 1
Figure 1. Performance of general VLMs on rare-scene remote sensing images. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. RRS-10K is organized into a hierarchical taxonomy comprising [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Overview of the RRS-10K Benchmark Construction. All remote sensing imagery is sourced from publicly available datasets and undergoes a rigorous [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Illustration of the SDFS. SDFS ranks candidate options according [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative examples of fine-grained tasks in RRS-10K, organized by Reasoning, Perception, and Robustness dimensions. Green text indicates GT [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 8
Figure 8. Figure 8: Robustness Task Visualization. The text in green indicates the GT [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 7
Figure 7. Figure 7: Reasoning Task Visualization. We incorporate weak supervision [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 9
Figure 9. Figure 9: Robustness dimension performance heatmap across 43 VLMs. Models are arranged as columns (sorted by average score), with four metrics as rows: [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 5 linked inside Pith

  1. [1]

    Creating xBD: A dataset for assessing building damage from satellite imagery,

    R. Guptaet al., “Creating xBD: A dataset for assessing building damage from satellite imagery,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), 2019, pp. 10–17

    2019

  2. [2]

    A biologist’s guide to the galaxy: Leveraging artificial intelligence and very high-resolution satellite imagery to monitor marine mammals from space,

    C. B. Khanet al., “A biologist’s guide to the galaxy: Leveraging artificial intelligence and very high-resolution satellite imagery to monitor marine mammals from space,”J. Mar . Sci. Eng., vol. 11, no. 3, 2023, art. no. 595

    2023

  3. [3]

    Military image captioning for low-altitude UA V or UGV perspectives,

    L. Pan, C. Song, X. Gan, K. Xu, and Y . Xie, “Military image captioning for low-altitude UA V or UGV perspectives,”Drones, vol. 8, no. 9, 2024, art. no. 421

    2024

  4. [4]

    Fine-grained interpretation of remote sensing image: A review,

    D. Wang, Z. Yan, and P. Liu, “Fine-grained interpretation of remote sensing image: A review,”Remote Sens., vol. 17, no. 23, 2025, art. no. 3887

    2025

  5. [5]

    A survey on deep learning-driven remote sensing image scene understanding: Scene classification, scene retrieval and scene-guided object detection,

    Y . Gu, Y . Wang, and Y . Li, “A survey on deep learning-driven remote sensing image scene understanding: Scene classification, scene retrieval and scene-guided object detection,”Appl. Sci., vol. 9, no. 10, 2019, art. no. 2110

    2019

  6. [6]

    Deep learning for remote sensing image scene classification: A review and meta-analysis,

    A. Thapa, T. Horanont, B. Neupane, and J. Aryal, “Deep learning for remote sensing image scene classification: A review and meta-analysis,” Remote Sens., vol. 15, no. 19, 2023, art. no. 4804

    2023

  7. [7]

    Object detection in optical remote sensing images: A survey and a new benchmark,

    K. Li, G. Wan, G. Cheng, L. Meng, and J. Han, “Object detection in optical remote sensing images: A survey and a new benchmark,”ISPRS J. Photogramm. Remote Sens., vol. 159, pp. 296–307, 2020

    2020

  8. [8]

    Oriented object detection in optical remote sensing images using deep learning: A survey,

    K. Wanget al., “Oriented object detection in optical remote sensing images using deep learning: A survey,”Artif. Intell. Rev., vol. 58, 2025, art. no. 350

    2025

Show all 60 references
  1. [9]

    A review of remote sensing image segmentation by deep learning methods,

    J. Li, Y . Cai, Q. Li, M. Kou, and T. Zhang, “A review of remote sensing image segmentation by deep learning methods,”Int. J. Digit. Earth, vol. 17, no. 1, 2024, art. no. 2328827

    2024

  2. [10]

    Deep learning-based semantic segmentation of remote sensing images: A review,

    J. Lv, Q. Shen, M. Lv, Y . Li, L. Shi, and P. Zhang, “Deep learning-based semantic segmentation of remote sensing images: A review,”Front. Ecol. Evol., vol. 11, 2023, art. no. 1201125

    2023

  3. [11]

    Learning transferable visual models from natural language supervision,

    A. Radfordet al., “Learning transferable visual models from natural language supervision,” inProc. 38th Int. Conf. Mach. Learn. (ICML), 2021, pp. 8748–8763

    2021

  4. [12]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 36, 2023, pp. 34 892– 34 916

    2023

  5. [13]

    RS-CLIP: Zero-shot remote sensing scene classification via contrastive vision-language supervision,

    X. Li, C. Wen, Y . Hu, and N. Zhou, “RS-CLIP: Zero-shot remote sensing scene classification via contrastive vision-language supervision,”Int. J. Appl. Earth Obs. Geoinf., vol. 124, 2023, art. no. 103497

    2023

  6. [14]

    RemoteCLIP: A vision-language foundation model for remote sensing,

    F. Liu, D. Chen, Z. Guan, X. Zhou, J. Zhu, and J. Zhou, “RemoteCLIP: A vision-language foundation model for remote sensing,”IEEE Trans. Geosci. Remote Sens., vol. 62, pp. 1–16, 2024

    2024

  7. [15]

    EarthGPT: A universal multi-modal large language model for multi-sensor image comprehension in remote sensing domain,

    W. Zhang, M. Cai, T. Zhang, Z. Yin, and X. Mao, “EarthGPT: A universal multi-modal large language model for multi-sensor image comprehension in remote sensing domain,”IEEE Trans. Geosci. Remote Sens., vol. 62, pp. 1–20, 2024

    2024

  8. [16]

    SkyEyeGPT: Unifying remote sensing vision-language tasks via instruction tuning with large language model,

    Y . Zhan, Z. Xiong, and Y . Yuan, “SkyEyeGPT: Unifying remote sensing vision-language tasks via instruction tuning with large language model,” ISPRS J. Photogramm. Remote Sens., vol. 221, pp. 64–77, 2025

    2025

  9. [17]

    RSVQA: Visual question answering for remote sensing data,

    S. Lobry, D. Marcos, J. Murray, and D. Tuia, “RSVQA: Visual question answering for remote sensing data,”IEEE Trans. Geosci. Remote Sens., vol. 58, no. 12, pp. 8555–8566, 2020

    2020

  10. [18]

    EarthVQA: Towards queryable Earth via relational reasoning-based remote sensing visual question answering,

    J. Wang, Z. Zheng, Z. Chen, A. Ma, and Y . Zhong, “EarthVQA: Towards queryable Earth via relational reasoning-based remote sensing visual question answering,” inProc. AAAI Conf. Artif. Intell., 2024, pp. 5481– 5489

    2024

  11. [19]

    VRSBench: A versatile vision- language benchmark dataset for remote sensing image understanding,

    X. Li, J. Ding, and M. Elhoseiny, “VRSBench: A versatile vision- language benchmark dataset for remote sensing image understanding,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), Datasets and Bench- marks Track, 2024, pp. 3229–3242

    2024

  12. [20]

    RSVLM-QA: A benchmark dataset for remote sensing vision language model-based question answering,

    X. Ziet al., “RSVLM-QA: A benchmark dataset for remote sensing vision language model-based question answering,” inProc. 33rd ACM Int. Conf. Multimedia (MM), 2025, pp. 12 905–12 911

    2025

  13. [21]

    VLRS-Bench: A vision- language reasoning benchmark for remote sensing,

    Z. Luo, D. Wang, H. Guo, J. Zhang, and B. Du, “VLRS-Bench: A vision- language reasoning benchmark for remote sensing,”arXiv preprint arXiv:2602.07045, 2026

    2026 arXiv

  14. [22]

    OmniEarth: A benchmark for evaluating vision-language models in geospatial tasks,

    R. Fuet al., “OmniEarth: A benchmark for evaluating vision-language models in geospatial tasks,”arXiv preprint arXiv:2603.09471, 2026

    2026

  15. [23]

    Long-tailed effect study in remote sensing semantic segmentation based on graph kernel principles,

    W. Cuiet al., “Long-tailed effect study in remote sensing semantic segmentation based on graph kernel principles,”Remote Sens., vol. 16, no. 8, 2024, art. no. 1398

    2024

  16. [24]

    MAR20: A benchmark for military aircraft recognition in remote sensing images,

    W. Yuet al., “MAR20: A benchmark for military aircraft recognition in remote sensing images,”Natl. Remote Sens. Bull., vol. 27, no. 12, pp. 2688–2696, 2023, (in Chinese)

    2023

  17. [25]

    A high resolution optical satellite image dataset for ship recognition and some new baselines,

    Z. Liu, L. Yuan, L. Weng, and Y . Yang, “A high resolution optical satellite image dataset for ship recognition and some new baselines,” in Proc. 6th Int. Conf. Pattern Recognit. Appl. Methods (ICPRAM), 2017, pp. 324–331

    2017

  18. [26]

    Construction and validation of remote sensing image dataset for fine-grained detection of military vehicles,

    D. Bai, Y . Yu, L. Song, B. Cheng, and H. Gao, “Construction and validation of remote sensing image dataset for fine-grained detection of military vehicles,”J. Image Graph., vol. 29, no. 12, pp. 3564–3577, 2024, (in Chinese)

    2024

  19. [27]

    A deep learning SAR target classification experiment on MSTAR dataset,

    C. I. Coman and R. Thaens, “A deep learning SAR target classification experiment on MSTAR dataset,” inProc. 19th Int. Radar Symp. (IRS), 2018, pp. 1–6

    2018

  20. [28]

    Multi-class geospatial object detection and geographic image classification based on collection of part detectors,

    G. Cheng, J. Han, P. Zhou, and L. Guo, “Multi-class geospatial object detection and geographic image classification based on collection of part detectors,”ISPRS J. Photogramm. Remote Sens., vol. 98, pp. 119–132, 2014

    2014

  21. [29]

    DOTA: A large-scale dataset for object detection in aerial images,

    G.-S. Xiaet al., “DOTA: A large-scale dataset for object detection in aerial images,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018, pp. 3974–3983

    2018

  22. [30]

    SAMChat: Introducing chain-of-thought reasoning and GRPO to a multimodal small language model for small- scale remote sensing,

    A. K ¨oksal and A. A. Alatan, “SAMChat: Introducing chain-of-thought reasoning and GRPO to a multimodal small language model for small- scale remote sensing,”IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 19, pp. 795–804, 2025

    2025

  23. [31]

    CHOICE: Benchmarking the remote sensing capabilities of large vision-language models,

    X. An, J. Sun, Z. Gui, and W. He, “CHOICE: Benchmarking the remote sensing capabilities of large vision-language models,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), Datasets and Benchmarks Track, 2025. IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERV ATIONS AND ...

    2025

  24. [32]

    Exploring models and data for remote sensing image caption generation,

    X. Lu, B. Wang, X. Zheng, and X. Li, “Exploring models and data for remote sensing image caption generation,”IEEE Trans. Geosci. Remote Sens., vol. 56, no. 4, pp. 2183–2195, 2018

    2018

  25. [33]

    Multimodal large models driven SAR image captioning: A benchmark dataset and baselines,

    Z. Gao, S. Sun, M.-M. Cheng, Y . Liu, and L. Liu, “Multimodal large models driven SAR image captioning: A benchmark dataset and baselines,”IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 18, pp. 24 011–24 026, 2025

    2025

  26. [34]

    FSAR-Cap: A fine- grained two-stage annotated dataset for SAR image captioning,

    J. Zhang, L. Cong, Z. Di, L. Zhang, and B. Zou, “FSAR-Cap: A fine- grained two-stage annotated dataset for SAR image captioning,”IEEE Geosci. Remote Sens. Lett., vol. 23, pp. 1–5, 2026

    2026

  27. [35]

    RS- LLaV A: A large vision-language model for joint captioning and question answering in remote sensing imagery,

    Y . Bazi, L. Bashmal, M. M. Al Rahhal, R. Ricci, and F. Melgani, “RS- LLaV A: A large vision-language model for joint captioning and question answering in remote sensing imagery,”Remote Sens., vol. 16, no. 9, 2024, art. no. 1477

    2024

  28. [36]

    SAR ship detection dataset (SSDD): Official release and comprehensive data analysis,

    T. Zhanget al., “SAR ship detection dataset (SSDD): Official release and comprehensive data analysis,”Remote Sens., vol. 13, no. 18, 2021, art. no. 3690

    2021

  29. [37]

    MilChat: A large language model and application for military equipment,

    X. Liu, J. Liu, P. Zhu, and T. Xiang, “MilChat: A large language model and application for military equipment,” inProc. 7th Int. Conf. Mach. Learn. Nat. Lang. Process. (MLNLP), 2024, pp. 1–5

    2024

  30. [38]

    Battlefield situation awareness using pretrained generative LLM,

    H. Chung, S. Hyun, and Y . G. Ha, “Battlefield situation awareness using pretrained generative LLM,” inProc. IEEE Int. Conf. Big Data Smart Comput. (BigComp), 2024, pp. 397–398

    2024

  31. [39]

    Fine-tuning and evaluating open-source large language models for the army domain,

    D. C. Ruiz and J. Sell, “Fine-tuning and evaluating open-source large language models for the army domain,”arXiv preprint arXiv:2410.20297, 2024

    2024 arXiv

  32. [40]

    Deep semantic understanding of high resolution remote sensing image,

    B. Qu, X. Li, D. Tao, and X. Lu, “Deep semantic understanding of high resolution remote sensing image,” inProc. Int. Conf. Comput. Inf. Telecommun. Syst. (CITS), 2016, pp. 1–5

    2016

  33. [41]

    GEOBench-VLM: Benchmarking vision-language models for geospatial tasks,

    M. S. Danishet al., “GEOBench-VLM: Benchmarking vision-language models for geospatial tasks,” inProc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2025, pp. 7132–7142

    2025

  34. [42]

    RSGPT: A remote sensing vision language model and benchmark,

    Y . Hu, J. Yuan, C. Wen, X. Lu, and X. Li, “RSGPT: A remote sensing vision language model and benchmark,”ISPRS J. Photogramm. Remote Sens., vol. 224, pp. 272–286, 2025

    2025

  35. [43]

    GeoChat: Grounded large vision-language model for remote sensing,

    K. Kuckreja, M. S. Danish, M. Naseer, A. Das, S. Khan, and F. S. Khan, “GeoChat: Grounded large vision-language model for remote sensing,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024, pp. 27 831–27 840

    2024

  36. [44]

    LHRS-Bot: Em- powering remote sensing with VGI-enhanced large multimodal language model,

    D. Muhtar, Z. Li, F. Gu, X. Zhang, and P. Xiao, “LHRS-Bot: Em- powering remote sensing with VGI-enhanced large multimodal language model,” inProc. Eur . Conf. Comput. Vis. (ECCV), 2024, pp. 440–457

    2024

  37. [45]

    Introducing GPT-5.4,

    OpenAI, “Introducing GPT-5.4,” [Online]. Available: https://openai.com/index/introducing-gpt-5-4/, Mar. 2026, Accessed: May 7, 2026

    2026

  38. [46]

    Grounding DINO: Marrying DINO with grounded pre- training for open-set object detection,

    S. Liuet al., “Grounding DINO: Marrying DINO with grounded pre- training for open-set object detection,” inProc. Eur . Conf. Comput. Vis. (ECCV), 2024, pp. 38–55

    2024

  39. [47]

    SAM 3: Segment anything with concepts,

    N. Carionet al., “SAM 3: Segment anything with concepts,”arXiv preprint arXiv:2511.16719, 2025

    2025 arXiv

  40. [48]

    Generating plausible distractors for multiple- choice questions via student choice prediction,

    Y . Lee, S. Kim, and Y . Jo, “Generating plausible distractors for multiple- choice questions via student choice prediction,” inProc. 63rd Annu. Meeting Assoc. Comput. Linguistics (ACL), 2025, pp. 23 669–23 692

    2025

  41. [49]

    BLEU: A method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: A method for automatic evaluation of machine translation,” inProc. 40th Annu. Meeting Assoc. Comput. Linguistics (ACL), 2002, pp. 311–318

    2002

  42. [50]

    ROUGE: A package for automatic evaluation of summaries,

    C.-Y . Lin, “ROUGE: A package for automatic evaluation of summaries,” inText Summarization Branches Out: Proc. ACL-04 Workshop, 2004, pp. 74–81

    2004

  43. [51]

    BERTScore: Evaluating text generation with BERT,

    T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “BERTScore: Evaluating text generation with BERT,” inProc. Int. Conf. Learn. Represent. (ICLR), 2020

    2020

  44. [52]

    The PASCAL visual object classes (VOC) challenge,

    M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zis- serman, “The PASCAL visual object classes (VOC) challenge,”Int. J. Comput. Vis., vol. 88, no. 2, pp. 303–338, 2010

    2010

  45. [53]

    Natural language object retrieval,

    R. Hu, H. Xu, M. Rohrbach, J. Feng, K. Saenko, and T. Darrell, “Natural language object retrieval,” inProc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016, pp. 4555–4564

    2016

  46. [54]

    Fully convolutional networks for semantic segmentation,

    E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 4, pp. 640–651, 2017

    2017

  47. [55]

    Measures of the amount of ecologic association between species,

    L. R. Dice, “Measures of the amount of ecologic association between species,”Ecology, vol. 26, no. 3, pp. 297–302, 1945

    1945

  48. [56]

    Gemini 3 Flash,

    Google DeepMind, “Gemini 3 Flash,” [Online]. Available: https://deepmind.google/models/gemini/flash/, Dec. 2025, Accessed: May 7, 2026

    2025

  49. [57]

    Qwen3-VL technical report,

    S. Baiet al., “Qwen3-VL technical report,”arXiv preprint arXiv:2511.21631, 2025

    2025 arXiv

  50. [58]

    GPT-4o system card,

    OpenAI, “GPT-4o system card,” [Online]. Available: https://openai.com/index/gpt-4o-system-card/, Aug. 2024, Accessed: May 7, 2026

    2024

  51. [59]

    Qwen2.5-VL technical report,

    S. Baiet al., “Qwen2.5-VL technical report,”arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  52. [60]

    Claude Sonnet 4.6 system card,

    Anthropic, “Claude Sonnet 4.6 system card,” [Online]. Available: https://www.anthropic.com/claude-sonnet-4-6-system-card, Feb. 2026, Accessed: May 7, 2026. Yuqiao Laireceived the B.S. degree in military science from the National University of Defense Technology, Nanjing, China...

    2026

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.