Pith. sign in

REVIEW 4 major objections 5 minor 35 references

Large Models in Dialogue for Active Perception and Anomaly Detection

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that engaging an LLM and a VQA model in a dialogue that controls a drone raises caption quality and anomaly detection accuracy in four simulated aerial monitoring scenes.

desk verdict A coherent extension of LLM-VQA dialogue to drone active perception, with a load-bearing but fixable evaluation flaw: the caption-quality metric is computed by the same model that generates the captions. read the letter →

arxiv 2501.16300 v1 pith:RGJTU6JM submitted 2025-01-27 cs.CV cs.AI

Tzoulio Chamiti , Nikolaos Passalis , Anastasios Tefas This is my paper · ORCID
classification cs.CVcs.AI
keywords ActiveperceptionAnomalydetectionLargelanguagemodelsVisualquestionansweringAerialmonitoringDronenavigationZero-shotlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that active perception in aerial monitoring can be built entirely from two off-the-shelf models conversing: a large language model (LLM) that issues drone movement commands and exploratory questions, and a visual question-answering (VQA) model that answers and captions each new image. The authors argue that this dialogue lets the system inspect a scene from multiple viewpoints and produce a richer, validated description, and that the same loop can detect anomalies such as fires and car crashes without any model fine-tuning. In four simulated environments, the proposed method is reported to raise the average caption-image matching score over a static baseline and to boost anomaly detection accuracy from around 0.20-0.53 to 0.73-0.90. The significance, if true, is a practical recipe for zero-shot active perception: pretrained language and vision components can be wired together with prompts and simple movement commands to guide exploration.

What carries the argument

The mechanism that carries the argument is the model dialogue loop, written as Q = f(A, C) for the LLM and (A, C) = g(Q, I) for the VQA network: the LLM maps the previous answer and caption to a new question and movement command, and the VQA maps that question and the current image to a new answer and caption. The named components are the Plug-and-Play VQA (PnP-VQA) framework, which uses BLIP for captioning and question answering and GradCAM to focus on relevant image patches, and GPT-3.5 as the LLM controller. What this loop does is convert one static image into a sequence of deliberately chosen viewpoints, then re-check claimed facts from new poses during validation before composing the final scene description, safety rules, and attention maps.

What would settle it

Run the same dialogue pipeline in the same four scenes, but have the final validated captions scored by a different vision-language model or by human annotators who do not see the PnP-VQA scores; if the proposed method no longer beats the static baseline, the reported gains are an artifact of self-scoring.

Watch

Extended reading notes

Core claim

The central claim is that two deep learning models can perform active perception and anomaly detection by taking turns in a dialogue. The LLM acts as the controller: it receives the VQA model's previous answer and caption, then outputs a new exploratory question and a movement command chosen from a fixed set (move closer, move back, move left/right, save position, ask a question, I know enough). The VQA model acts as the sensor: given the question and the new drone image, it returns an answer, a caption, and a caption-image matching score. The process runs in three phases: active perception, validation (revisiting saved positions with added pose noise and ensembling the answers), and explanation (GradCAM attention maps that show which image regions support the conclusions). The authors report that this pipeline improves caption-image matching scores in all four environments and raises anomaly detection accuracy from a 0.20-0.53 baseline to 0.73-0.90, while an anomaly-triggered early-stop rule cuts the average experiment time from roughly 12 minutes to under 5 minutes.

Load-bearing premise

The paper's quantitative evidence assumes that the image-caption matching score produced by the PnP-VQA model is an independent, unbiased measure of caption quality and anomaly detection success, even though the same model generates the captions being scored.

Editorial extensions

If this is right

  • If the reported gains hold, drones can monitor unfamiliar scenes in a zero-shot fashion, without retraining, by reusing general-purpose language and vision models.
  • The same template of the LLM as controller and the VQA model as sensor, with saved poses revisited for validation, should transfer to other robots, including ground vehicles and pan-tilt cameras.
  • The anomaly-triggered early-stop rule makes the system practical for time-critical hazard response, reducing average exploration time from about 12 minutes to under 5 minutes.
  • Attention maps generated during validation give a human operator a concrete account of which image regions led to a caption or hazard alert.
  • The paper's evaluation suggests that dialogue-based active perception can outperform static perception even without any domain-specific training data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the caption-image matching scores come from the same PnP-VQA model that writes the captions, the numerical gains may partly reflect the model agreeing with itself; scoring the final captions with an independent vision-language model or human raters would test this.
  • The prompt trick of framing the task as a game to suppress hallucination is a reusable design pattern: explicitly constraining an LLM's role and world model can improve reliability in other LLM-controlled systems.
  • The validation phase's noisy repositioning and answer ensembling effectively create a small, automatically labeled dataset of question-image-answer triples, which could be used to fine-tune a more specialized perception model.
  • A direct next experiment is to run the same dialogue loop on real drone footage and compare the validated captions against human-written descriptions of the same scenes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes an active-perception and anomaly-detection framework for a simulated drone. A GPT-3.5 controller issues natural-language movement commands and exploratory questions; a PnP-VQA/BLIP module answers and captions images; positions are saved, revisited with noise during validation, and captions/descriptions are ensembled. Experiments in four AirSim environments report caption-image matching scores and anomaly detection accuracies for baseline versus proposed method, along with qualitative captions, descriptions, and attention maps.

Significance. If the reported gains were measured against an independent standard, the contribution would be useful: zero-shot integration of an LLM controller with a VQA sensor, active perception via dialogue, and explainable attention maps. The authors also release code and prompts, which supports reproducibility. However, the current evaluation does not establish the central claim because the headline metric is generated and scored by the same model, and the anomaly criterion is subjective. The qualitative examples and attention maps provide partial support but do not compensate for the lack of an external benchmark.

major comments (4)
  1. [Section 4, Table 1] The caption-image matching score is computed by the PnP-VQA model, which is the same model whose BLIP captioning module generates the captions being scored. The score is also used inside the pipeline to select and validate captions (Section 3). Therefore the improvement from baseline to proposed (e.g., 0.384 to 0.585 for Mountain Landscape) may reflect the model's stylistic self-consistency rather than improved perception. The authors should add an independent evaluation, such as human ratings, an external vision-language scorer, or standard captioning metrics against ground-truth descriptions, and report per-run variance.
  2. [Section 4, Table 2] Anomaly detection success is defined as the anomaly "appears in captions in a coherent and grammatically logical manner," but no operational protocol is given for judging coherence and logicality. Without a rubric, ground-truth anomaly labels, or inter-annotator agreement, the reported accuracies are not reproducible and cannot be compared across methods. Please define detection as a specific mention of the injected hazard type (fire, car crash, etc.), or use human evaluation with multiple annotators, and report false positives.
  3. [Section 4, Tables 1 and 2] All quantitative results are averages over ten runs with no standard deviations, confidence intervals, or statistical tests. Given the small number of runs and the stochasticity of LLM sampling, navigation noise, and caption sampling, the reader cannot assess whether the differences are meaningful. Please report per-run results, variance, and significance tests.
  4. [Section 4, Table 4 and accompanying text] The paper concedes that "hallucinations can still occur despite the validation process," and the final descriptions contain fabricated elements (e.g., "computer artwork" in Mountain Landscape with fire, "digital object" in Public Square, "bush tucker" in Lake with fire). These examples show that the validation module does not reliably filter hallucinated content, which weakens the claim that the proposed method outputs a validated, detailed scene description. The authors should quantify hallucination rates and show, under an external metric, that final descriptions are more accurate than baseline captions.
minor comments (5)
  1. [Section 3] The notation f(A,C) and g(Q,I) is confusing because A is used both for the answer sequence and for the LLM input sequence; please introduce distinct symbols for the LLM input context and the VQA answer.
  2. [Section 4] Report the exact prompt templates and the random seed handling in the paper or supplementary material; the repository link is helpful but the paper should state which parts of the experiments are deterministic.
  3. [Section 5] The claim that "there is currently no other established way to implement and evaluate active perception in unstructured open-world setups" is too strong; please qualify it to the authors' knowledge and cite existing benchmarks for embodied question answering and active perception.
  4. [Tables 3 and 4] The green/red/orange highlighting may not be visible in grayscale printing; please add textual markers or a legend.
  5. [Figure 4] The four environment images are not labeled in the text; consider labeling the subfigures to make references clear.

Circularity Check

1 steps flagged · score 6.0 of 10

The quantitative claim is largely measured by the same PnP-VQA/BLIP image-caption matching score that the method itself uses to select, validate, and ensemble captions, so the reported gains in Tables 1 and 2 partly reflect model self-consistency rather than an independent test of perception.

  1. self definitional [Section 3 (Proposed Method) and Section 4 (Experimental Evaluation), validation-mode description and Table 1 evaluation protocol.]
    "Additionally, the VQA model outputs an image-caption matching score in order to help the LLM distinguish between good and bad captions. ... we also save the question-image pairs which hold the highest matching score percentage for explanation mode. ... To quantitatively evaluate the performance of the proposed method we compute the caption-image matching score (using the VQA model) at the drone’s spawn position and at every subsequent position revisited during the validation module."

    The same PnP-VQA/BLIP matching score is used both as the internal caption-quality signal and as the external evaluation metric. During validation the pipeline keeps question-image pairs with the highest matching-score percentage and ensembles the final description from them; the Section 4 evaluation then averages that same score over the revisited positions and compares it with the baseline's initial caption. Since the proposed captions are therefore selected and updated to maximize this very score, the improvement in Table 1 is in part guaranteed by construction and measures the model's self-consistency rather than independently verified perceptual accuracy.

full rationale

The paper's central mechanism—an LLM-driven dialogue with active drone movement—has genuine independent content and is not itself defined in terms of the evaluation. However, the only quantitative evidence for improved perception is the caption-image matching score computed by the same BLIP/PnP-VQA model that generates, filters, and validates the captions. Specifically, Section 3 states that the matching score helps the LLM distinguish good from bad captions, and validation explicitly saves the highest-matching question-image pairs; Section 4 then uses that identical score as the evaluation metric. This makes the Table 1 gains substantially self-referential. The anomaly-detection scores in Table 2 rely on the same model-generated captions with a subjective, non-operationalized success criterion, and the paper itself concedes that hallucinations still occur after validation. No external benchmark, human evaluation protocol, or independently labeled anomaly set is reported. While this is not a case of fitted parameters being relabeled as predictions, and no external benchmark is needed for the system demonstration, the load-bearing quantitative claim is partially circular; hence a score of 6 is appropriate.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No fitted constants or invented entities appear; the framework is an integration of existing models. The main hand-chosen degrees of freedom are the movement step sizes, validation noise, and unreported decoding settings. The evaluation additionally assumes the self-produced matching score is an honest measure, which is a domain assumption rather than a fitted parameter.

free parameters (3)
  • Movement step sizes = 10 m forward, 10 m left, 10 m right, 5 m backward
    Hand-chosen in the active perception command set (Section 3); they determine how much of the scene is covered and therefore the reported caption matching and detection scores.
  • Validation position noise = Gaussian noise, magnitude not specified
    Random Gaussian noise added to saved positions to create new question-image pairs during validation (Section 3); the noise level affects diversity and the final matched scores.
  • LLM decoding hyperparameters = not reported
    GPT-3.5 output variability is not controlled or reported, yet the dialogue trajectory and anomaly detection depend on sampling behavior.
assumptions (4)
  • domain assumption AirSim is a high-fidelity simulation whose visual and physical feedback transfers to real-world aerial monitoring.
    All experiments are conducted in AirSim (Section 4); conclusions about real drones and in-the-wild adaptation rely on this transfer assumption.
  • domain assumption PnP-VQA caption-image matching score is a valid proxy for description quality.
    The primary quantitative metric in Table 1 is computed by the VQA model itself; no external human annotation or third-party metric is used.
  • domain assumption An anomaly is successfully detected when the model's caption mentions it in a coherent, grammatically logical way.
    Section 4 defines detection this way; the criterion is judged within the pipeline's own outputs and lacks an independent rubric.
  • domain assumption The conversational prompt structure and game framing prevent or sufficiently limit LLM hallucination.
    Section 3 introduces the game prompt to prevent hallucination, but Section 4 concedes hallucinations still occur despite validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Large Models in Dialogue for Active Perception and Anomaly Detection." pith.science (2026). https://pith.science/paper/RGJTU6JM

@misc{pith2026250116300,
  author       = {Pith},
  title        = {Pith review of: Large Models in Dialogue for Active Perception and Anomaly Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RGJTU6JM}},
  note         = {Machine review of arXiv:2501.16300}
}
read the original abstract

Autonomous aerial monitoring is an important task aimed at gathering information from areas that may not be easily accessible by humans. At the same time, this task often requires recognizing anomalies from a significant distance or not previously encountered in the past. In this paper, we propose a novel framework that leverages the advanced capabilities provided by Large Language Models (LLMs) to actively collect information and perform anomaly detection in novel scenes. To this end, we propose an LLM based model dialogue approach, in which two deep learning models engage in a dialogue to actively control a drone to increase perception and anomaly detection accuracy. We conduct our experiments in a high fidelity simulation environment where an LLM is provided with a predetermined set of natural language movement commands mapped into executable code functions. Additionally, we deploy a multimodal Visual Question Answering (VQA) model charged with the task of visual question answering and captioning. By engaging the two models in conversation, the LLM asks exploratory questions while simultaneously flying a drone into different parts of the scene, providing a novel way to implement active perception. By leveraging LLMs reasoning ability, we output an improved detailed description of the scene going beyond existing static perception approaches. In addition to information gathering, our approach is utilized for anomaly detection and our results demonstrate the proposed methods effectiveness in informing and alerting about potential hazards.

Figures

Figures reproduced from arXiv: 2501.16300 by the authors.

Figure 1
Figure 1. Overview of the proposed model dialogue approach. First a drone cap [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 1
Figure 1. First, the drone leverages a VQA model which provides descriptions of the [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. A typical example of the operation of the proposed method. During active [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figures from the paper (6 more)
Figure 3
Figure 3. Figure 3: The employed VQA architecture. the drone with a detailed prompt consisting of a set of commands mapped to a specific function apiece, certain rules the GPT3.5 outputs must follow, the general goal of the task and tips on how to filter and extract information from capti…
Figure 2
Figure 2. Figure 2: As a result, when the drone returns to its starting position it is able [PITH_FULL_IMAGE:figures/full_fig_p008_2.png]
Figure 4
Figure 4. Figure 4: To quantitatively evaluate the performance of the proposed method we com￾pute the caption-image matching score (using the VQA model) at the drone’s spawn position and at every subsequent position revisited during the validation [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 4
Figure 4. Figure 4: Four different environments were used for the conducted experiments: a [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Example anomalies in the four different environments. Note that some [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Two examples for two different questions, indicating the additional explain [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 16 canonical work pages

  1. [1]

    Drone-surveillance for search and rescue in natural disaster,

    B. Mishra, D. Garg, P. Narang, and V. Mishra, “Drone-surveillance for search and rescue in natural disaster,”Computer Communications, vol. 156, pp. 1–10, 2020

    work page 2020

  2. [2]

    Uav-based surveillance sys- tem: an anomaly detection approach,

    A. Chriki, H. Touati, H. Snoussi, and F. Kamoun, “Uav-based surveillance sys- tem: an anomaly detection approach,” inProceedings of the IEEE Symposium on Computers and Communications (ISCC) , 2020, pp. 1–6

    work page 2020

  3. [3]

    Anomaly detection, localization and classification for railway inspection,

    R. Gasparini, A. D’Eusanio, G. Borghi, S. Pini, G. Scaglione, S. Calderara, E. Fedeli, and R. Cucchiara, “Anomaly detection, localization and classification for railway inspection,” inProceedings of the International Conference on Pattern Recognition (ICPR), 2021, pp. 3419–3426. 14 T. Chamiti et al. Table 4: We showcase our methods ability to provide desc...

    work page 2021

  4. [4]

    Smart Autopilot Drone System for Surface Surveillance and Anomaly Detection via Customizable Deep Neural Network,

    X. Zhai, K. Liu, W. Nash, and D. Castineira, “Smart Autopilot Drone System for Surface Surveillance and Anomaly Detection via Customizable Deep Neural Network,” ser. IPTC International Petroleum Technology Conference, vol. Day 2 Tue, January 14, 2020, 01 2020, p. D021S053R001

    work page 2020

  5. [5]

    An Autonomous Drone Surveil- lance and Tracking Architecture,

    E. Unlu, E. Zenou, N. Riviere, and P.-E. Dupouy, “An Autonomous Drone Surveil- lance and Tracking Architecture,” in 2019 Autonomous Vehicles and Machines Large Models in Dialogue for Active Perception and Anomaly Detection 15 Conference, A VM 2019, vol. 2019, Jan. 2019, pp. 35–1–35–7

    work page 2019

  6. [6]

    Revisiting active perception,

    R. Bajcsy, Y. Aloimonos, and J. K. Tsotsos, “Revisiting active perception,”Au- tonomous Robots, vol. 42, pp. 177–196, 2018

    work page 2018

  7. [7]

    How to select and use tools?: Active perception of target objects using multimodal deep learning,

    N. Saito, T. Ogata, S. Funabashi, H. Mori, and S. Sugano, “How to select and use tools?: Active perception of target objects using multimodal deep learning,”IEEE Robotics and Automation Letters , vol. 6, no. 2, pp. 2517–2524, 2021

    work page 2021

  8. [8]

    Enabling high-resolution pose estimation in real time using active perception,

    T. Manousis, N. Passalis, and A. Tefas, “Enabling high-resolution pose estimation in real time using active perception,” in Proceedings of the IEEE International Conference on Image Processing (ICIP) , 2023, pp. 2425–2429

    work page 2023

Show all 35 references
  1. [9]

    Vqa: Visual question answering,

    A. Agrawal, J. Lu, S. Antol, M. Mitchell, C. L. Zitnick, D. Batra, and D. Parikh, “Vqa: Visual question answering,”arXiv:1505.00468, 2016

    2016 arXiv

  2. [10]

    Mqa: Answering the question via robotic manipulation,

    Y. Deng*, D. Guo*, X. Guo, N. Zhang, H. Liu, and F. Sun, “Mqa: Answering the question via robotic manipulation,” in Robotics: Science and Systems XVII , ser. RSS2021. Robotics: Science and Systems Foundation, 2021

    2021

  3. [11]

    Iqa: Visual question answering in interactive environments,

    D. Gordon, A. Kembhavi, M. Rastegari, J. Redmon, D. Fox, and A. Farhadi, “Iqa: Visual question answering in interactive environments,”arXiv:1712.03316, 2018

    2018 arXiv

  4. [12]

    Embodied question answering,

    A. Das, S. Datta, G. Gkioxari, S. Lee, D. Parikh, and D. Batra, “Embodied question answering,” arXiv:1711.11543, 2017

    2017 arXiv

  5. [13]

    Chatgpt for robotics: Design principles and model abilities,

    S. Vemprala, R. Bonatti, A. Bucker, and A. Kapoor, “Chatgpt for robotics: Design principles and model abilities,”arXiv:2306.17582, 2023

    2023 arXiv

  6. [14]

    From words to flight: Integrating openai chatgpt with px4/gazebo for natural language-based drone control,

    M. Lamine, . Tazir, M. Mancas, and T. Dutoit, “From words to flight: Integrating openai chatgpt with px4/gazebo for natural language-based drone control,”Pro- ceedings of the 13th International Workshop on Computer Science and Engineering , 2023

    2023

  7. [15]

    Improved trust in human-robot collaboration with chatgpt,

    Y. Ye, H. You, and J. Du, “Improved trust in human-robot collaboration with chatgpt,” arXiv:2304.12529, 2023

    2023 arXiv

  8. [16]

    Code as policies: Language model programs for embodied control,

    J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng, “Code as policies: Language model programs for embodied control,” arXiv:2209.07753, 2023

    2023 arXiv

  9. [17]

    Visual chatgpt: Talking, drawing and editing with visual foundation models,

    C. Wu, S. Yin, W. Qi, X. Wang, Z. Tang, and N. Duan, “Visual chatgpt: Talking, drawing and editing with visual foundation models,”arXiv:2303.04671, 2023

    2023 arXiv

  10. [18]

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,

    Y. Shen, K. Song, X. Tan, D. Li, W. Lu, and Y. Zhuang, “Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,”arXiv:2303.17580, 2023

    2023 arXiv

  11. [19]

    Next-gpt: Any-to-any multimodal llm,

    S. Wu, H. Fei, L. Qu, W. Ji, and T.-S. Chua, “Next-gpt: Any-to-any multimodal llm,” arXiv:2309.05519, 2023

    2023 arXiv

  12. [20]

    Cliport: What and where pathways for robotic manipulation,

    M. Shridhar, L. Manuelli, and D. Fox, “Cliport: What and where pathways for robotic manipulation,” arXiv:2109.12098, 2021

    2021 arXiv

  13. [21]

    Reshap- ing robot trajectories using natural language commands: A study of multi-modal data alignment using transformers,

    A. Bucker, L. Figueredo, S. Haddadin, A. Kapoor, S. Ma, and R. Bonatti, “Reshap- ing robot trajectories using natural language commands: A study of multi-modal data alignment using transformers,”arXiv:2203.13411, 2022

    2022 arXiv

  14. [22]

    Language-conditioned imitation learning for robot manipulation tasks,

    S. Stepputtis, J. Campbell, M. Phielipp, S. Lee, C. Baral, and H. B. Amor, “Language-conditioned imitation learning for robot manipulation tasks,” arXiv:2010.12083, 2020

    2010 arXiv

  15. [23]

    Chatgpt asks, blip-2 answers: Automatic questioning towards enriched visual descriptions,

    D. Zhu, J. Chen, K. Haydarov, X. Shen, W. Zhang, and M. Elhoseiny, “Chatgpt asks, blip-2 answers: Automatic questioning towards enriched visual descriptions,” arXiv:2303.06594, 2023

    2023 arXiv

  16. [24]

    Fusecap: Leveraging large language models for enriched fused image captions,

    N. Rotstein, D. Bensaid, S. Brody, R. Ganz, and R. Kimmel, “Fusecap: Leveraging large language models for enriched fused image captions,”arXiv:2305.17718, 2023

    2023 arXiv

  17. [25]

    Chatting makes perfect: Chat-based image retrieval,

    M. Levy, R. Ben-Ari, N. Darshan, and D. Lischinski, “Chatting makes perfect: Chat-based image retrieval,”arXiv:2305.20062, 2023. 16 T. Chamiti et al

    2023 arXiv

  18. [26]

    Machine-to-machine visual dialoguing with chatgpt for enriched textual image description,

    R. Ricci, Y. Bazi, and F. Melgani, “Machine-to-machine visual dialoguing with chatgpt for enriched textual image description,”Remote Sensing, vol. 16, no. 3, 2024

    2024

  19. [27]

    Promptcap: Prompt- guided task-aware image captioning,

    Y. Hu, H. Hua, Z. Yang, W. Shi, N. A. Smith, and J. Luo, “Promptcap: Prompt- guided task-aware image captioning,”arXiv:2211.09699, 2023

    2023 arXiv

  20. [28]

    Prophet: Prompting large lan- guage models with complementary answer heuristics for knowledge-based visual question answering,

    Z. Yu, X. Ouyang, Z. Shao, M. Wang, and J. Yu, “Prophet: Prompting large lan- guage models with complementary answer heuristics for knowledge-based visual question answering,” arXiv:2303.01903, 2023

    2023 arXiv

  21. [29]

    Vlc-bert: Visual question answering with contextualized commonsense knowledge,

    S. Ravi, A. Chinchure, L. Sigal, R. Liao, and V. Shwartz, “Vlc-bert: Visual question answering with contextualized commonsense knowledge,”arXiv:2210.13626, 2022

    2022 arXiv

  22. [30]

    Plug-and-play vqa: Zero-shot vqa by conjoining large pretrained models with zero training,

    A. M. H. Tiong, J. Li, B. Li, S. Savarese, and S. C. H. Hoi, “Plug-and-play vqa: Zero-shot vqa by conjoining large pretrained models with zero training,” arXiv:2210.08773, 2023

    2023 arXiv

  23. [31]

    Blip: Bootstrapping language-imagepre-training forunifiedvision-languageunderstandingandgeneration,

    J. Li, D.Li, C.Xiong, andS. Hoi,“Blip: Bootstrapping language-imagepre-training forunifiedvision-languageunderstandingandgeneration,” arXiv:2201.12086,2022

    2022 arXiv

  24. [32]

    Grad-cam: Visual explanations from deep networks via gradient-based localiza- tion,

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localiza- tion,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 618–626

    2017

  25. [33]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Nee- lakantan, P. Shyam, G. Sastry, A. Askellet al. , “Language models are few-shot learners,” Proceedings of the Advances in Neural Information Processing Systems , vol. 33, pp. 1877–1901, 2020

    1901

  26. [34]

    Siren’s song in the ai ocean: A survey on hallucination in large language models,

    Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen, L. Wang, A. T. Luu, W. Bi, F. Shi, and S. Shi, “Siren’s song in the ai ocean: A survey on hallucination in large language models,”arXiv:2309.01219, 2023

    2023 arXiv

  27. [35]

    Airsim: High-fidelity visual and phys- ical simulation for autonomous vehicles,

    S. Shah, D. Dey, C. Lovett, and A. Kapoor, “Airsim: High-fidelity visual and phys- ical simulation for autonomous vehicles,”arXiv:1705.05065, 2017

    2017 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.