Pith. sign in

REVIEW 4 major objections 5 minor 46 references

Visual Question Answering on Multiple Remote Sensing Image Modalities

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multispectral and radar context improve visual question answering on remote sensing images.

desk verdict A genuinely new three-modality RSVQA dataset with a careful automated pipeline, but the headline multi-modal gain is concentrated in geo-context question types and the experimental evaluation needs a re-work. read the letter →

arxiv 2505.15401 v1 pith:MCMVY54G submitted 2025-05-21 cs.CV

Hichem Boussaid , Lucrezia Tosato , Flora Weissgerber , Camille Kurtz , Laurent Wendling , Sylvain Lobry This is my paper · ORCID
classification cs.CV
keywords visualquestionansweringremotesensingmultimodalfusionmultispectralimagerysyntheticapertureradarBERTTAMMIdatasetmulti-resolution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces TAMMI, a visual question answering dataset in which each question refers to a very high resolution RGB orthophoto plus a wider multispectral patch and a radar patch of the same scene, and proposes MM-RSVQA, a VisualBERT-based model that fuses the three modalities. The central claim is that the extra spectral and radar context improves answer accuracy even though the questions concern only the area covered by the RGB patch: MM-RSVQA reaches 65.56% average accuracy against 61.94% for the RGB-only ablation. The paper also argues that the dataset's automated question-generation pipeline, built from public vector geodatabases, makes it easy to extend to new regions and modalities. A sympathetic reader would take the contribution as a first demonstration that multi-resolution, multi-modal context is useful for remote sensing VQA, opening a task that existing RGB-only benchmarks do not cover.

What carries the argument

The central mechanism is a trainable fusion of four feature streams: a frozen ResNet-152 pretrained on ImageNet for the 20 cm RGB patch, two ResNet-50 encoders pretrained on BigEarthNet for the 10-band multispectral patch and the VV/VH/ratio radar channels, and DistilBERT token embeddings for the question. These are concatenated with a [SEP] separator and processed by VisualBERT, whose transformer layers align visual tokens with text; a linear classifier then predicts among the 1,000 most frequent answers. The context size for the multispectral and radar patches is a tunable hyperparameter ($L_{MS}=100$, $L_{SAR}=200$ in the experiments), which is what lets the model see beyond the VHR extent.

What would settle it

A concrete check would be an ablation where the MS and SAR patches are randomly shifted to a nearby location while the question and RGB patch stay fixed: if MM-RSVQA retains most of its accuracy, the 'context' gain is not visual. A complementary check is a question-only baseline with no images: if it approaches 65.56% on department, region, or flood questions, the benchmark's answers are predictable without imagery.

Watch

Extended reading notes

Core claim

On TAMMI, answering questions from all three modalities outperforms every ablation. Average accuracy rises from 61.94% (VHR only) to 65.56%, and overall accuracy from 52.22% to 55.11%. The largest gains appear in classification questions such as department (89.78 to 98.96), urban class (69.20 to 93.78), water body type (81.10 to 89.38), and flood type (90.60 to 97.91). The authors interpret this as evidence that spatial context from wider multispectral and radar patches helps even when the question is restricted to the RGB patch's geographical extent, and that SAR and multispectral data carry complementary information. The paper presents this as a new task, not a mature solution, and notes that spatial questions such as absolute location, area, and relative location remain difficult.

Load-bearing premise

The ground-truth answers come from external vector geodatabases rather than from direct image inspection, and the paper assumes those databases are correct, that the image patches actually contain the information needed for each answer, and that the accuracy gains come from visual context rather than from geographic or linguistic regularities in the dataset.

Editorial extensions

If this is right

  • If the central claim is right, remote sensing VQA should be built on multi-resolution inputs: wider context patches, not just the visible RGB crop, are needed for classification and flood-type questions.
  • The result validates the choice made in earlier datasets to use Sentinel-2 RGB channels, since adding all 10 bands gives only a small gain over RGB alongside VHR; but it also shows Sentinel-1 radar can be substituted for multispectral context with comparable accuracy.
  • The automated pipeline implies new TAMMI extensions can be built for new departments or new sensors without manual question writing, so the task can grow beyond the three French regions.
  • Because MS+SAR without VHR already reaches 65.15% average accuracy, the very high resolution modality is not indispensable for many question types; a model could choose modalities dynamically.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the authors do not run is a control where the MS and SAR patches are taken from a different location: if accuracy stays high, the gain would come from dataset or geographic leakage rather than visual context.
  • The same VisualBERT fusion recipe could be carried to medical VQA, where the paper itself suggests multi-modality matters; the dataset pipeline's cropping logic would need to be replaced by registered scans rather than georeferenced tiles.
  • The strong department and urban classification gains suggest the model may be learning geographic priors from context; future work should report per-question accuracy conditioned on whether the answer is actually visible in the RGB patch.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces a new multi-modal, multi-resolution VQA dataset for remote sensing (TAMMI), coupling very high-resolution RGB orthophotos, Sentinel-2 multispectral data, and Sentinel-1 SAR images with automatically generated question/answer pairs derived from external vector geo-databases. The authors also propose a baseline model, MM-RSVQA, based on VisualBERT, which fuses features from the three image modalities and the question text. Experiments on TAMMI report an average accuracy of 65.56% for MM-RSVQA versus 61.94% for a VHR-only model, and the paper interprets this as evidence that additional MS/SAR context improves VQA performance. The dataset and code are made publicly available.

Significance. The paper offers a potentially valuable resource: a large, extensible dataset with diverse question types and modalities, and an automated pipeline for generating question/answer pairs. The public release of data and code is a concrete contribution that could facilitate future work on multi-modal remote sensing VQA. If the central empirical claim were robustly established, the work would support the value of combining complementary sensors. However, the current experimental evidence is preliminary and confounded; the reported improvement is concentrated in question types that are answerable from geographic context, and the ablation design does not separate the effect of increased spatial context from the effect of genuinely multi-spectral/SAR information. As such, the paper's main claim is plausible but not yet convincingly demonstrated.

major comments (4)
  1. [Section 5.3, Table 2] The central claim that MS and SAR context improves VQA is confounded by the difference in spatial extent. The VHR patches cover 200m x 200m, while the MS and SAR patches are larger (L_MS=100, L_SAR=200, providing roughly 1km or more of context). The ablation row 'MS RGB + VHR' uses the same larger MS context but only the RGB bands of Sentinel-2, and it already achieves 64.72% AA versus 61.94% for VHR-only, capturing most of the gain of the full MM-RSVQA (65.56%). This indicates that the improvement is largely due to providing a wider geographical footprint, not to the spectral or SAR-specific information. To support the stated conclusion that multi-spectral and SAR data 'enhanc[e] the ability of the model to understand and classify complex features,' the authors need an additional control: a VHR-only model with a comparable field of view (e.g., a 1km VHR patch) or an experiment that varies context size while holding the sensor modality fixed.
  2. [Section 5.3, Table 2] The reported improvement is not uniformly distributed across question types. Recomputing from Table 2, the nine geo/context-based types (Flood Presence, Mountain Presence, Flood Level, Flood Type, Land Cover, Urban, Department, Region, Mountain Name) show a gain of about +6.6 AA points (from 87.08 to 93.67), whereas the remaining twelve visually grounded types (Presence, Count, Density, Area, Percentage, Absolute Location, Water, Vegetation, Distance, Comparison, Relative Location, Nearest) improve by only about +1.4 AA points (from 43.09 to 44.48). Because the latter gain is small and the paper reports no error bars, multiple seeds, or significance tests, the evidence that multi-modal context improves performance on visually grounded VQA is weak. The authors should either restrict their central claim to context-sensitive geographic questions or provide additional analysis (e.g., confidence intervals, per-type significance tests) to demonstrate that the 1.4-point gap is not noise.
  3. [Section 5.1] The handling of test answers outside the top-k vocabulary is unspecified. The paper states that training uses only questions whose answer is in the top 1000 most frequent answers (covering 86.6% of training answers), but evaluation is performed on all test samples. It is not described what happens when a test ground-truth answer is not in the model's output vocabulary: is the question discarded, counted as an error, or mapped to an 'other' class? This choice directly affects the reported accuracy. The authors must clarify the exact evaluation protocol and report the coverage of the top-1000 answer set on the test split.
  4. [Section 5.1] The experimental evaluation is preliminary: a single random split, no error bars, no multiple seeds, and no comparison to any existing RSVQA model. TAMMI is a new dataset, so direct comparison with prior benchmarks is not possible, but the authors could adapt at least one established RSVQA method (e.g., the CNN+RNN model of [21] or the LXMERT-based approach of [11]) to the multi-modal setting. Without such a baseline, it is unclear whether MM-RSVQA is a competent baseline or simply an arbitrary architecture. Adding a small number of comparative experiments would substantially strengthen the paper.
minor comments (5)
  1. [Section 3.2] There is a typo: 'question/answer parirs' should be 'question/answer pairs'.
  2. [Section 3.3] The balancing formulas (N_Q,A = N_P / min(N_A,10) and N_P / log(x+3)) are introduced without a formal equation number; please number them and provide a brief justification for the empirical constants.
  3. [Section 4.1] The description of the SAR feature extractor is unclear: a ResNet-50 pretrained on BigEarthNet is used for SAR patches, but BigEarthNet is a Sentinel-2 multispectral benchmark. Please explain how the three SAR channels (VV, VH, ratio) are fed to this network and whether any adaptation of the pretrained weights was performed.
  4. [Section 5.3] The statement that 'it is the first time that this hypothesis is experimentally demonstrated for VQA' (regarding RGB-only vs. 10-band MS) is too strong; the paper only demonstrates it on one dataset with one architecture. Please temper the claim.
  5. [Section 3.3] The dataset construction uses external vector databases (BDTopo, TRI, BU20, CLC, EMA) for ground-truth answers. The paper should explicitly acknowledge that many question types (Department, Region, Urban, Flood) are inherently geo-context questions rather than purely visual reasoning questions, and should discuss the implications for the interpretation of the results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports a supervised-learning benchmark and does not derive its central claim from fitted parameters or load-bearing self-citations.

full rationale

The paper's central claim is that adding MS and SAR context to VHR imagery improves remote sensing VQA (Section 5.3, Table 2). This is an empirical result obtained by training MM-RSVQA on the TAMMI dataset and evaluating on a held-out test split; no equation in the paper defines the reported accuracy in terms of the training labels, and no fitted parameter is renamed as a prediction. The question/answer pairs are generated from external vector databases (BDTopo, TRI, BU20, CLC, EMA) independently of the model, and the per-question-type accuracies are computed on a test set not used for training or hyperparameter selection. Self-citations (RSVQA [21], RSVQAxBEN [22], SAR-RSVQA [31]) are used as background and dataset comparisons, not as the source of the reported numbers. A possible validity concern, that gains are concentrated in question types answerable from large geographic context rather than from visual reasoning, is a correctness or interpretation issue and not a circularity of the kind where the derivation reduces to its own inputs. Under the stated standard (quote a specific reduction or fitted-input rename), no circular step is present.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on several hand-set hyperparameters (patch sizes, answer candidate count, balancing constants) and on domain assumptions about the correctness of geo-database labels, SAR alignment, transferability of frozen features, and the validity of closed-set evaluation. No new physical entities are introduced; the 'new' element is the dataset and task formulation.

free parameters (4)
  • L_MS (multispectral context patch size) = 100
    Section 5.1 sets L_MS=100 as the spatial size of Sentinel-2 patches; this hyperparameter controls the amount of surrounding context and directly affects the multi-modal fusion claim.
  • L_SAR (SAR context patch size) = 200
    Section 5.1 sets L_SAR=200 for Sentinel-1 patches; chosen by the authors, not optimized, and influences how much radar context the model sees.
  • top_k answer candidate set = 1000
    Section 5.1: training restricted to the 1000 most frequent answers, covering 86.6% of training answers; this dimensionality reduction changes the task into closed-set classification and affects all accuracy numbers.
  • Dataset balancing formula constants = N_P/min(N_A,10) and N_P/log(x+3)
    Section 3.3: the automatic balancing of answer distributions uses these formulas, with x+3 chosen empirically to cap frequent numeric answers; they shape the dataset's label distribution.
assumptions (5)
  • domain assumption The external geo-databases (BDTopo, TRI, BU20, CLC, EMA) provide correct ground-truth answers for each VHR patch
    Section 3.3: all question/answer pairs are derived from these vector databases; if they are inaccurate, the dataset labels are wrong.
  • domain assumption SAR patches are correctly coregistered and debursted so that the three modalities are spatially aligned
    Section 3.2 and supplementary material: the pipeline relies on LabSAR projection and debursting; misalignment would make the MS/SAR features uninformative or misleading.
  • domain assumption Frozen ResNet features pretrained on ImageNet and BigEarthNet transfer sufficiently to the three modalities
    Section 5.1: feature extractors are frozen; if the representations do not capture relevant visual cues, the model cannot succeed regardless of fusion.
  • domain assumption Treating VQA as closed-set classification over the top 1000 answers is a valid evaluation scheme
    Section 5.1: the model outputs one of 1000 classes, but the test set contains all answers; the mismatch is not explained.
  • domain assumption The questions are answerable from the provided image modalities
    Section 3.3: many answers are geographic facts (department, region, urban class) that may be visible only through surrounding context; the paper assumes the image content carries the answer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visual Question Answering on Multiple Remote Sensing Image Modalities." pith.science (2026). https://pith.science/paper/MCMVY54G

@misc{pith2026250515401,
  author       = {Pith},
  title        = {Pith review of: Visual Question Answering on Multiple Remote Sensing Image Modalities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MCMVY54G}},
  note         = {Machine review of arXiv:2505.15401}
}
read the original abstract

The extraction of visual features is an essential step in Visual Question Answering (VQA). Building a good visual representation of the analyzed scene is indeed one of the essential keys for the system to be able to correctly understand the latter in order to answer complex questions. In many fields such as remote sensing, the visual feature extraction step could benefit significantly from leveraging different image modalities carrying complementary spectral, spatial and contextual information. In this work, we propose to add multiple image modalities to VQA in the particular context of remote sensing, leading to a novel task for the computer vision community. To this end, we introduce a new VQA dataset, named TAMMI (Text and Multi-Modal Imagery) with diverse questions on scenes described by three different modalities (very high resolution RGB, multi-spectral imaging data and synthetic aperture radar). Thanks to an automated pipeline, this dataset can be easily extended according to experimental needs. We also propose the MM-RSVQA (Multi-modal Multi-resolution Remote Sensing Visual Question Answering) model, based on VisualBERT, a vision-language transformer, to effectively combine the multiple image modalities and text through a trainable fusion process. A preliminary experimental study shows promising results of our methodology on this challenging dataset, with an accuracy of 65.56% on the targeted VQA task. This pioneering work paves the way for the community to a new multi-modal multi-resolution VQA task that can be applied in other imaging domains (such as medical imaging) where multi-modality can enrich the visual representation of a scene. The dataset and code are available at https://tammi.sylvainlobry.com/.

Figures

Figures reproduced from arXiv: 2505.15401 by the authors.

Figure 1
Figure 1. Summary of our contributions. We introduce a new task [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Geographical extent of the TAMMI dataset, covering se [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Graphical outline of the proposed MM-RSVQA ( [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Sentinel-1 SLC images before (a) and after (b) the ap [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 41 canonical work pages

  1. [21]

    RSVQA: Visual question answering for remote sensing data

    Sylvain Lobry, Diego Marcos, Jesse Murray, and Devis Tuia. RSVQA: Visual question answering for remote sensing data. TGRS, 58(12):8555–8566, 2020. 1, 2, 3, 6, 7

    work page 2020

  2. [11]

    Cross- modal visual question answering for remote sensing data

    Rafael Felix, Boris Repasky, Samuel Hodge, Reza Zolfaghari, Ehsan Abbasnejad, and Jamie Sherrah. Cross- modal visual question answering for remote sensing data. In DICTA, pages 1–9. IEEE, 2021. 2

    work page 2021

  3. [1]

    Visual Question An- swering for Wishart H-Alpha Classification of Polarimetric SAR Images

    Hossein Aghababaei and Alfred Stein. Visual Question An- swering for Wishart H-Alpha Classification of Polarimetric SAR Images. InIGARSS, pages 11231–11234. IEEE, 2024. 3

    work page 2024

  4. [2]

    Bottom-up and top-down attention for image captioning and visual question answering

    Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. InCVPR Proceedings, pages 6077–6086, 2018. 2

    work page 2018

  5. [3]

    VQA: Visual question answering

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. VQA: Visual question answering. InICCV Proceedings, pages 2425–2433, 2015. 1, 2, 7

    work page 2015

  6. [4]

    Language trans- formers for remote sensing visual question answering

    Christel Chappuis, Vincent Mendez, Eliot Walt, Sylvain Lo- bry, Bertrand Le Saux, and Devis Tuia. Language trans- formers for remote sensing visual question answering. In IGARSS, pages 4855–4858. IEEE, 2022. 2

    work page 2022

  7. [5]

    Prompt-RSVQA: Prompting visual context to a language model for remote sensing visual question answering

    Christel Chappuis, Val ´erie Zermatten, Sylvain Lobry, Bertrand Le Saux, and Devis Tuia. Prompt-RSVQA: Prompting visual context to a language model for remote sensing visual question answering. InCVPR, pages 1372– 1381, 2022. 2

    work page 2022

  8. [6]

    Multi-task prompt-RSVQA to explicitly count objects on aerial images

    Christel Chappuis, Charlotte Sertic, Nicola Santacroce, Javiera Castillo Navarro, Sylvain Lobry, Bertrand Le Saux, and Devis Tuia. Multi-task prompt-RSVQA to explicitly count objects on aerial images. InBMVC Workshop, 2023. 2

    work page 2023

Show all 46 references
  1. [7]

    The curse of lan- guage biases in remote sensing VQA: the role of spatial at- tributes, language diversity, and the need for clear evaluation

    Christel Chappuis, Eliot Walt, Vincent Mendez, Sylvain Lo- bry, Bertrand Le Saux, and Devis Tuia. The curse of lan- guage biases in remote sensing VQA: the role of spatial at- tributes, language diversity, and the need for clear evaluation. arXiv preprint arXiv:2311.16782, 2023. 3

    2023 arXiv

  2. [8]

    BERT: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018. 2

    2018 arXiv

  3. [9]

    PubMedCLIP: How Much Does CLIP Benefit Visual Ques- tion Answering in the Medical Domain? InEACL, pages 1151–1163, 2023

    Sedigheh Eslami, Christoph Meinel, and Gerard De Melo. PubMedCLIP: How Much Does CLIP Benefit Visual Ques- tion Answering in the Medical Domain? InEACL, pages 1151–1163, 2023. 2

    2023

  4. [10]

    S2 missionhttps : / / sentiwiki.copernicus.eu/web/s2- mission# S2Mission - RadiometricPerformanceS2 - Mission - Radiometric - Performancetrue

    European Space Agency. S2 missionhttps : / / sentiwiki.copernicus.eu/web/s2- mission# S2Mission - RadiometricPerformanceS2 - Mission - Radiometric - Performancetrue. (Accessed 14/11/2024). 3

    2024

  5. [12]

    Making the V in VQA matter: El- evating the role of image understanding in visual question answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Ba- tra, and Devi Parikh. Making the V in VQA matter: El- evating the role of image understanding in visual question answering. InCVPR Proceedings, pages 6904–6913, 2017. 5

    2017

  6. [13]

    Overview of image- CLEF 2018 medical domain visual question answering task

    Sadid A Hasan, Yuan Ling, Oladimeji Farri, Joey Liu, Hen- ning M ¨uller, and Matthew Lungren. Overview of image- CLEF 2018 medical domain visual question answering task. Proceeding of CLEF, 2018. 1

    2018

  7. [14]

    PromptCap: Prompt-guided image captioning for SAR with GPT-3

    Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah A Smith, and Jiebo Luo. PromptCap: Prompt-guided image captioning for SAR with GPT-3. InICCV Proceedings, pages 2963–2975, 2023. 1

    2023

  8. [15]

    CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning

    Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning. InCVPR Proceedings, pages 2901–2910, 2017. 4

    2017

  9. [16]

    Q: How to specialize large vision-language models to data-scarce VQA tasks? a: Self-train on unlabeled images! InCVPR Proceedings, pages 15005–15015, 2023

    Zaid Khan, Vijay Kumar BG, Samuel Schulter, Xiang Yu, Yun Fu, and Manmohan Chandraker. Q: How to specialize large vision-language models to data-scarce VQA tasks? a: Self-train on unlabeled images! InCVPR Proceedings, pages 15005–15015, 2023. 1, 2

    2023

  10. [17]

    Deep learning in multi- modal remote sensing data fusion: A comprehensive review

    Jiaxin Li, Danfeng Hong, Lianru Gao, Jing Yao, Ke Zheng, Bing Zhang, and Jocelyn Chanussot. Deep learning in multi- modal remote sensing data fusion: A comprehensive review. JAG, 112:102926, 2022. 2

    2022

  11. [18]

    Visualbert: A simple and perfor- mant baseline for vision and language.arXiv preprint arXiv:1908.03557, 2019

    Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and perfor- mant baseline for vision and language.arXiv preprint arXiv:1908.03557, 2019. 6

    1908 arXiv

  12. [19]

    A comprehensive study of GPT-4V’s multimodal capabilities in medical imaging

    Yingshu Li, Yunyi Liu, Zhanyu Wang, Xinyu Liang, Lingqiao Liu, Lei Wang, Leyang Cui, Zhaopeng Tu, Longyue Wang, and Luping Zhou. A comprehensive study of GPT-4V’s multimodal capabilities in medical imaging. medRxiv, pages 2023–11, 2023. 1

    2023

  13. [20]

    Medical visual question answering: A survey.Artificial In- telligence in Medicine, page 102611, 2023

    Zhihong Lin, Donghao Zhang, Qingyi Tao, Danli Shi, Gho- lamreza Haffari, Qi Wu, Mingguang He, and Zongyuan Ge. Medical visual question answering: A survey.Artificial In- telligence in Medicine, page 102611, 2023. 1, 2

    2023

  14. [22]

    RSVQA meets BigEarthNet: a new, large-scale, visual question an- swering dataset for remote sensing

    Sylvain Lobry, Beg ¨um Demir, and Devis Tuia. RSVQA meets BigEarthNet: a new, large-scale, visual question an- swering dataset for remote sensing. InIGARSS, pages 1218–

  15. [23]

    Deep learning and earth observation to support the sustainable development goals: Current approaches, open challenges, and future opportunities.GRS, 10(2):172–200,

    Claudio Persello, Jan Dirk Wegner, Ronny H ¨ansch, Devis Tuia, Pedram Ghamisi, Mila Koeva, and Gustau Camps- Valls. Deep learning and earth observation to support the sustainable development goals: Current approaches, open challenges, and future opportunities.GRS, 10(2):172–200,

  16. [24]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InICML, pages 8748–8763. PMLR, 2021. 2

    2021

  17. [25]

    DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108, 2019

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108, 2019. 6

    1910 arXiv

  18. [26]

    How much can clip benefit vision-and-language tasks?arXiv preprint arXiv:2107.06383, 2021

    Sheng Shen, Liunian Harold Li, Hao Tan, Mohit Bansal, Anna Rohrbach, Kai-Wei Chang, Zhewei Yao, and Kurt Keutzer. How much can clip benefit vision-and-language tasks?arXiv preprint arXiv:2107.06383, 2021. 2

    2021 arXiv

  19. [27]

    BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding

    Gencer Sumbul, Marcela Charfuelan, Beg ¨um Demir, and V olker Markl. BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding. In IGARSS, pages 5901–5904. IEEE, 2019. 6

    2019

  20. [28]

    BigEarthNet-MM: A large-scale, multimodal, multilabel benchmark archive for remote sensing image classification and retrieval [software and data sets].GRS, 9(3):174–180, 2021

    Gencer Sumbul, Arne De Wall, Tristan Kreuziger, Filipe Marcelino, Hugo Costa, Pedro Benevides, Mario Caetano, Beg¨um Demir, and V olker Markl. BigEarthNet-MM: A large-scale, multimodal, multilabel benchmark archive for remote sensing image classification and retrieval [softwar...

    2021

  21. [29]

    LXMERT: Learning cross- modality encoder representations from transformers

    Hao Tan and Mohit Bansal. LXMERT: Learning cross- modality encoder representations from transformers. In EMNLP-IJCNLP. Association for Computational Linguis- tics, 2019. 2

    2019

  22. [30]

    Segmentation-guided attention for visual question answering from remote sensing images

    Lucrezia Tosato, Hichem Boussaid, Flora Weissgerber, Camille Kurtz, Laurent Wendling, and Sylvain Lobry. Segmentation-guided attention for visual question answering from remote sensing images. InIGARSS, pages 2750–2754. IEEE, 2024. 2

    2024

  23. [31]

    Can SAR improve RSVQA performance? In EUSAR, pages 1287–1292

    Lucrezia Tosato, Sylvain Lobry, Flora Weissgerber, and Lau- rent Wendling. Can SAR improve RSVQA performance? In EUSAR, pages 1287–1292. VDE, 2024. 2, 3, 6

    2024

  24. [32]

    A visual question answering method for SAR ship: Breaking the requirement for multimodal dataset construction and model fine-tuning.arXiv preprint arXiv:2411.01445, 2024

    Fei Wang, Chengcheng Chen, Hongyu Chen, Yugang Chang, and Weiming Zeng. A visual question answering method for SAR ship: Breaking the requirement for multimodal dataset construction and model fine-tuning.arXiv preprint arXiv:2411.01445, 2024. 3

    2024 arXiv

  25. [33]

    Labsar, a one- gcp coregistration tool for sar–insar local analysis in high- mountain regions.Frontiers in Remote Sensing, 3:935137,

    Flora Weissgerber, Laurane Charrier, Cyril Thomas, Jean- Marie Nicolas, and Emmanuel Trouv ´e. Labsar, a one- gcp coregistration tool for sar–insar local analysis in high- mountain regions.Frontiers in Remote Sensing, 3:935137,

  26. [34]

    LabSAR, a one-GCP coregistration tool for SAR–InSAR local analysis in high- mountain regions.Frontiers in Remote Sensing, 3, 2022

    Flora Weissgerber, Laurane Charrier, Cyril Thomas, Jean- Marie Nicolas, and Emmanuel Trouv´e. LabSAR, a one-GCP coregistration tool for SAR–InSAR local analysis in high- mountain regions.Frontiers in Remote Sensing, 3, 2022. 4

    2022

  27. [35]

    Stacked attention networks for image question answering

    Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alex Smola. Stacked attention networks for image question answering. InCVPR Proceedings, pages 21–29, 2016. 2

    2016

  28. [36]

    Self- paced curriculum learning for visual question answering on remote sensing data

    Zhenghang Yuan, Lichao Mou, and Xiao Xiang Zhu. Self- paced curriculum learning for visual question answering on remote sensing data. InIGARSS, pages 2999–3002. IEEE,

  29. [37]

    Multi- lingual augmentation for robust visual question answering in remote sensing images

    Zhenghang Yuan, Lichao Mou, and Xiao Xiang Zhu. Multi- lingual augmentation for robust visual question answering in remote sensing images. InJURSE, pages 1–4. IEEE, 2023. 2

    2023

  30. [38]

    Frequency domain transfer learning for remote sensing visual question answering

    Enyuan Zhao, Ziyi Wan, Xinyue Liang, Min Ye, Jie Nie, Lei Huang, et al. Frequency domain transfer learning for remote sensing visual question answering. 2

  31. [39]

    Exploring data and models in SAR ship image captioning.IEEE Access, 10:pp

    Kai Zhao and Wei Xiong. Exploring data and models in SAR ship image captioning.IEEE Access, 10:pp. 91150–91159,

  32. [40]

    Mutual attention inception network for remote sensing visual question answering.TGRS, 60:1–14, 2021

    Xiangtao Zheng, Binqiang Wang, Xingqian Du, and Xiao- qiang Lu. Mutual attention inception network for remote sensing visual question answering.TGRS, 60:1–14, 2021. 2, 6

    2021

  33. [41]

    TRAR: Routing the attention spans in transformer for visual question answering

    Yiyi Zhou, Tianhe Ren, Chaoyang Zhu, Xiaoshuai Sun, Jianzhuang Liu, Xinghao Ding, Mingliang Xu, and Ron- grong Ji. TRAR: Routing the attention spans in transformer for visual question answering. InICCV Proceedings, pages 2074–2084, 2021. 2 Visual Question Answering on Multiple...

    2021

  34. [42]

    To do so, we need the geographical position of the center of the VHR patch

    We use the algorithm presented in [33] to find the posi- tion of the center of the VHR patch in the SAR image. To do so, we need the geographical position of the center of the VHR patch. This position must include altitude, due the geometrical distortions inherent in SAR, part...

  35. [43]

    The latitude and longitude of the VHR patches are extracted from the meta-data of the image. Using this latitude and longitude, the altitude is given by this https://geoservices.ign.fr/documentation/services/services- deprecies/calcul-altimetrique-rest and these information ar...

  36. [44]

    To find the correct swath, the projection of the geographical point is applied, using the meta-data linked to each swath

    The Sentinel-1 Single Look Complex (SLC) images are separated in three swath. To find the correct swath, the projection of the geographical point is applied, using the meta-data linked to each swath. The only swath for which the algorithm returns a valid position is selected

  37. [45]

    The S1 images need to be debursted (removing of the black line and of the overlap) to get a continuous image before to extract the S1 patch that is inputted in the model

    In each swath, the S1 image is divided in different over- lapping bursts, that are all stored in the same file sepa- rated by black lines. The S1 images need to be debursted (removing of the black line and of the overlap) to get a continuous image before to extract the S1 patc...

  38. [46]

    A tail- value elimination procedure is performed on each chan- nel separately using statistics information extracted over the whole dataset

    When a patch of sizeL SAR ×L SAR (LSAR specified by the user) is extracted, the two polarimetric channels are converted in dB, and the ratio is computed. A tail- value elimination procedure is performed on each chan- nel separately using statistics information extracted over t...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.