Pith. sign in

REVIEW 4 major objections 8 minor 1 cited by

Visual RAG: Expanding MLLM visual knowledge without fine-tuning

T0 review · 4 major / 8 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Retrieving the most similar labeled images from a knowledge base lets a multimodal LLM match or beat many-shot in-context learning for image classification while using only about a quarter of the demonstrating examples.

desk verdict Visual RAG is a clean, well-ablated port of text RAG to image classification, but its headline efficiency claim rests on uncontrolled baseline comparisons and post hoc selection; the accuracy story is plausible, the 23% number is not yet established. read the letter →

arxiv 2501.10834 v1 pith:JJEKAUR6 submitted 2025-01-18 cs.CV cs.AIcs.LG

Mirco Bonomo , Simone Bianco This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords VisualRAGretrieval-augmentedgenerationmultimodallargelanguagemodelsin-contextlearningimageclassificationCLIPFAISStraining-freeadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a multimodal large language model (MLLM) can expand its visual knowledge at inference time, without fine-tuning, by retrieving the most relevant labeled images from a knowledge base and giving them to the model as in-context examples. The proposed Visual RAG pipeline selects a small number of demonstrating images per query using CLIP embeddings and a FAISS nearest-neighbor index, then prompts Gemini 1.5 Pro to classify the query image with those examples as context. Across eight datasets spanning medical, remote-sensing, texture, fine-grained, and natural-image classification, the authors report that Visual RAG matches or exceeds many-shot in-context learning, the prior state of the art, gaining about two percentage points in average accuracy while using only about 23 percent of the demonstrating examples. If the claim holds, this offers a cheap, rapidly updatable alternative to fine-tuning for adapting MLLMs to new visual domains and tasks.

What carries the argument

The machinery is a three-stage retrieval-augmented pipeline adapted from text RAG to images. In the indexing stage, each labeled image in the demo set is encoded with CLIP and stored in a FAISS flat L2 index. In the retrieval stage, the query image's CLIP embedding is used to find the $k$ nearest knowledge-base images by Euclidean distance, and these images become the demonstrating examples. In the generation stage, Gemini 1.5 Pro is prompted with the retrieved image-label pairs plus the query image, and asked to return the class label in a fixed format. The load-bearing assumption is that Euclidean proximity in CLIP embedding space tracks classification-relevant similarity, so the retrieved images' labels are the right supervision for the query; the paper also validates the components through a retriever-only ablation, which labels queries by majority vote of neighbors, and a generator-only zero-shot baseline.

What would settle it

Measure the fraction of CLIP nearest neighbors whose label matches the query label on a given dataset; if that nearest-neighbor label purity is at or below the dataset's base class rate, Visual RAG should not outperform random many-shot in-context learning at the same example count.

Watch

Extended reading notes

Core claim

The central claim is that for MLLM image classification, the relevance of in-context demonstrations matters more than their quantity. By replacing the random selection used in many-shot in-context learning with nearest-neighbor retrieval in CLIP embedding space, Visual RAG achieves accuracy close to or higher than the many-shot baseline in six of eight datasets, with an average improvement of about two percentage points, while using on average only 23 percent of the demonstrating examples. The largest gains appear on FIVES, EuroSAT, and DTD, where the retriever's ability to discard irrelevant images reduces context noise. The two datasets where Visual RAG falls slightly short, TerraIncognita and OxfordPets, show gaps of 0.25 and 0.29 percentage points, which the authors attribute to the retriever focusing on background rather than subject and to fine-grained inter-class similarity respectively.

Load-bearing premise

The method depends on CLIP's L2 nearest neighbors in the knowledge base carrying the correct supervision for the query; when visual similarity is dominated by background scene or by a single dominant label, the retrieved examples can mislead the generator.

Editorial extensions

If this is right

  • Updating the knowledge base becomes a database operation: adding, removing, or relabeling images immediately changes the examples the MLLM sees, with no retraining step.
  • Token usage and inference cost for in-context adaptation drop by roughly three-quarters on average, since Visual RAG reaches the same or better accuracy with about 23 percent of the demonstrating examples.
  • The method generalizes across image domains and classification formats, covering medical imagery, remote sensing, texture, fine-grained pets, multi-label chest X-rays, and more, with the fewest examples needed on OxfordPets, DTD, and FIVES.
  • Accuracy continues to rise as more retrieved examples are added, so Visual RAG retains the scaling behavior of many-shot in-context learning while staying more efficient.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: Because the retriever is a generic CLIP model rather than a task-specific one, the same knowledge base and index could serve multiple downstream tasks by swapping the prompt, making Visual RAG a reusable knowledge-injection layer for MLLMs.
  • Inference: A retriever trained or adapted to emphasize class-discriminative features would likely close the TerraIncognita and CheXpert gaps and reduce the example count further; the paper identifies retrieval as the bottleneck but does not test such a retriever.
  • Inference: Part of the efficiency gain may stem from the 'lost in the middle' effect, since smaller contexts use the examples more evenly; the paper cites this effect but does not isolate it experimentally.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes Visual RAG, a training-free retrieval-augmented classification method for multimodal LLMs. For each query image, CLIP embeddings are used with FAISS (IndexFlatL2) to retrieve a small set of similar labeled images from a dataset-specific demo set; these are inserted as in-context examples into a Gemini 1.5 Pro prompt. The method is evaluated on eight classification datasets (HAM10000, FIVES, CheXpert, TerraIncognita, UCMerced, EuroSAT, Oxford Pets, DTD), comparing against a zero-shot baseline and the many-shot ICL results reported in Jiang et al. The headline claims are that Visual RAG matches or exceeds many-shot ICL accuracy (about +2% average) while using roughly 23% of the demonstrating examples on average, with no fine-tuning and reduced context/API cost. Ablations include a zero-shot generator-only condition and a retriever-only majority-vote condition.

Significance. If established, the result is practically significant: a retrieval-based demonstration selection that matches or improves on random many-shot ICL with dramatically fewer examples would offer a cheap, fine-tuning-free way to adapt MLLMs to new classification domains and would extend textual RAG ideas to the visual modality. The paper has strengths: the pipeline is simple and clearly described; the ablation design (generator-only, retriever-only, full system) is sensible; the choice of eight varied datasets is appropriate for a generalization claim; the authors are explicit about domains where retrieval fails (TerraIncognita, CheXpert, OxfordPets) and about the cost constraints that shaped the experiments. The verification infrastructure is not shipped (no code/data release is mentioned), but the method is straightforward to reimplement.

major comments (4)
  1. [Sec. 5, Table 2 and Fig. 4] The efficiency claim ('23% of examples on average', '+2% on average') is not supported by the reported protocol. The many-shot ICL numbers are taken from [17] and were not rerun under identical conditions; only 'a subset of baseline tests was performed' (Sec. 4.3). The comparison in Fig. 4 is a post hoc best-of-n contrast: for each dataset it selects the highest-accuracy many-shot run from [17] against the lowest-example Visual RAG run that matches or beats that number, with the closest run selected when no run matches. Without identical reruns, error bars, or repeated API evaluations, the reported accuracy gaps (several are under 0.3%) are within plausible API stochasticity and the 'match or beat' criterion can be satisfied by noise. The 23% average should be rederived under a fixed, pre-specified comparison protocol, e.g., rerunning the many-shot baselines under the same prompt/cost conditions and reporting matched-k or matched-accuracy comparisons with variance estimates.
  2. [Sec. 4.3 and Table 2] The CheXpert comparison is not apples-to-apples. The evaluation section states that CheXpert is scored with macro-averaged F1 (the multi-label task), but Table 2 lists the column header 'Accuracy' and the average mixes CheXpert's macro-F1 with accuracy values from the other seven single-label datasets. The paper should report the exact metric used for each dataset, use a consistent label in the table, and avoid averaging macro-F1 with accuracy. This matters because the CheXpert gain (42.43 vs 42.23) is among the smallest reported.
  3. [Sec. 5 (accuracy gaps)] Some of the strongest accuracy comparisons appear to compare different operating points. For datasets where 'Visual RAG did not achieve the performance levels of Many-shot ICL' (TerraIncognita 66.42 vs 66.67; OxfordPets 97.14 vs 97.43), the text dismisses the gaps as 'very low' (0.25% and 0.29%), yet for the headline '+2% average improvement' the same magnitude of difference is treated as meaningful. Either both gaps are within noise (in which case the +2% claim needs error bars) or both are meaningful (in which case the 0.25% deficits are also meaningful). The paper needs a consistent treatment of small gaps.
  4. [Sec. 5, Fig. 4 caption] The protocol used to select the compared points should be specified before looking at the results. As written, the caption and text describe a selection rule ('experiment with the lowest number of examples that was able to match or surpass that accuracy') that is data-dependent. For a fair efficiency comparison, the selection rule should be fixed a priori (e.g., per-dataset k equal to 5, 10, 50, or 100) and the efficiency ratio reported for that fixed rule, with the best-accuracy comparison reported separately.
minor comments (8)
  1. [Sec. 2] The related-work discussion of multimodal ICL would benefit from a clearer distinction between retrieval-based methods for LLMs and the few prior visual ICL works; several cited works ([13], [39]) are described only briefly.
  2. [Sec. 3.3] The prompt is referenced as 'detailed in Appendix A [17]' and the appendix reproduces the prompt from [17]; this is fine, but the paper should state explicitly that the prompt is not new and that its design was not tuned for the proposed method.
  3. [Sec. 4.2] The sentence 'we opted to test our solution on a wide variety of datasets ... rather than focusing on achieving peak performance' is somewhat ambiguous; it suggests the reported numbers may not be the method's ceiling, which is useful, but it should also state which runs (if any) were repeated or used for the final claims.
  4. [Table 2 caption] The 'Baseline (zero-shot)' column is also described as the 'generator-only ablation study' in the table caption and in Sec. 6; the dual labeling is clear but could be made explicit in the table caption.
  5. [Throughout] The paper uses 'demonstrating examples' where 'demonstration examples' is the standard term; consistency would improve readability.
  6. [Fig. 3] The plot ranges appear to start at zero and some curves are hard to separate; per-dataset accuracy ranges or a zoomed view would aid readability, especially for the small gaps at the top (UCMerced, OxfordPets).
  7. [Sec. 6] The retriever-only tie-breaking rule ('in the case of a tie between the correct and other class(es), the prediction is considered incorrect') is a fine design choice, but it should also state how ties among non-correct classes are broken.
  8. [Sec. 5, Fig. 5] The caption says 'Answer Choice' strings are shown as single words (e.g., 'Japanese chin'), which is fine, but the class-description strings used in the prompt are not specified; providing the exact prompt templates or a link to released code would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Visual RAG's accuracy and efficiency claims are empirical measurements against external benchmarks, not derived from their own inputs.

full rationale

The paper's central claim is an empirical comparison against external benchmarks on eight standard datasets. Visual RAG accuracy is measured on held-out test sets, the many-shot ICL baseline is taken from the independent prior work [17], and the retrieval mechanism (CLIP embeddings with FAISS L2 nearest-neighbor search) is a fixed, parameter-free procedure with no fitted parameters. The generator is a fixed API model. No equation in the paper defines the predicted accuracy in terms of the input or of a fitted parameter, and no load-bearing premise is justified only by a self-citation; the authors' own ablations (retriever-only, generator-only) compare against external test labels. The reported 23% example-efficiency figure is a post hoc selection of the lowest-k run that matches or surpasses the best baseline accuracy, which is a favorable analysis and a possible correctness concern, but it is not a circular reduction: the accuracy at that k is an independently measured quantity rather than an output determined by construction. The CheXpert metric inconsistency (macro-F1 in Section 4.3 vs. an Accuracy table in Table 2) is a reporting/correctness issue, not circularity. The paper is self-contained against external benchmarks, so the circularity score is 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The approach has no derived equations with fitted constants. One hand-chosen hyperparameter (number of retrieved examples) drives the reported efficiency numbers. The central assumptions are that CLIP embeddings capture classification-relevant similarity and that Gemini can learn from image-label pairs. No new entities are introduced.

free parameters (1)
  • number of retrieved examples k = 5, 10, 50, and 100 (dataset dependent)
    Chosen by hand for experiments; Figure 4 then selects the smallest k that reaches the baseline accuracy, which is a post hoc reporting choice rather than a fitted model parameter.
assumptions (3)
  • domain assumption CLIP embedding L2 distance is a valid proxy for classification-relevant visual similarity.
    Section 3.2 uses L2 nearest neighbors in CLIP space to choose demonstrating examples; all results depend on this, and Section 5 reports failures when background or dominant label dominates.
  • domain assumption Gemini 1.5 Pro can perform in-context learning from image-label pairs in the borrowed prompt format.
    Section 3.3 relies on the model's ICL ability; the method does not derive or verify this capability beyond the reported experiments.
  • domain assumption The demo set is a representative labeled sample of each target domain.
    Section 4.1 builds the knowledge base from demo sets; if a demo set is not representative, retrieved examples will mislead the generator.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visual RAG: Expanding MLLM visual knowledge without fine-tuning." pith.science (2026). https://pith.science/paper/JJEKAUR6

@misc{pith2026250110834,
  author       = {Pith},
  title        = {Pith review of: Visual RAG: Expanding MLLM visual knowledge without fine-tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JJEKAUR6}},
  note         = {Machine review of arXiv:2501.10834}
}
read the original abstract

Multimodal Large Language Models (MLLMs) have achieved notable performance in computer vision tasks that require reasoning across visual and textual modalities, yet their capabilities are limited to their pre-trained data, requiring extensive fine-tuning for updates. Recent researches have explored the use of In-Context Learning (ICL) to overcome these challenges by providing a set of demonstrating examples as context to augment MLLMs performance in several tasks, showing that many-shot ICL leads to substantial improvements compared to few-shot ICL. However, the reliance on numerous demonstrating examples and the limited MLLMs context windows presents significant obstacles. This paper aims to address these challenges by introducing a novel approach, Visual RAG, that synergically combines the MLLMs capability to learn from the context, with a retrieval mechanism. The crux of this approach is to ensure to augment the MLLM knowledge by selecting only the most relevant demonstrating examples for the query, pushing it to learn by analogy. In this way, relying on the new information provided dynamically during inference time, the resulting system is not limited to the knowledge extracted from the training data, but can be updated rapidly and easily without fine-tuning. Furthermore, this greatly reduces the computational costs for improving the model image classification performance, and augments the model knowledge to new visual domains and tasks it was not trained for. Extensive experiments on eight different datasets in the state of the art spanning several domains and image classification tasks show that the proposed Visual RAG, compared to the most recent state of the art (i.e., many-shot ICL), is able to obtain an accuracy that is very close or even higher (approx. +2% improvement on average) while using a much smaller set of demonstrating examples (approx. only 23% on average).

Figures

Figures reproduced from arXiv: 2501.10834 by the authors.

Figure 5
Figure 5. The results are relative to the case where five demonstra [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Seeing the Unseen: Towards Training-Free Inspection for Wind Turbine Blades Using Knowledge-Augmented Vision Language Models

    cs.CV 2025-10 conditional novelty 5.0 of 10

    A retrieval-augmented vision-language framework scored 30/30 on a four-class wind-turbine blade damage test, vs 28/30 for the same model without retrieval — a two-sample difference the paper's own confidence intervals...

Reference graph

Works this paper leans on

44 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [17]

    Jiang, J

    Y . Jiang, J. Irvin, J. H. Wang, M. A. Chaudhry, J. H. Chen, and A. Y . Ng. Many-shot in-context learning in multimodal foundation models,

    work page

  2. [1]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 tech- nical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [2]

    Barnett, S

    S. Barnett, S. Kurniawan, S. Thudumu, Z. Brannelly, and M. Abdel- razek. Seven failure points when engineering a retrieval augmented generation system, 2024. URL https://arxiv.org/abs/2401.05856

    arXiv 2024

  4. [3]

    Beery, G

    S. Beery, G. Van Horn, and P. Perona. Recognition in terra incognita. In Proceedings of the European conference on computer vision (ECCV), pages 456–473, 2018

    work page 2018

  5. [4]

    Brown, B

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language mod- els are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  6. [5]

    S. Chen, S. Wong, L. Chen, and Y . Tian. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595, 2023

    arXiv 2023

  7. [6]

    Cimpoi, S

    M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi. Describ- ing textures in the wild, 2013. URL https://arxiv.org/abs/1311.3618

    arXiv 2013

  8. [7]

    Q. Dong, L. Li, D. Dai, C. Zheng, J. Ma, R. Li, H. Xia, J. Xu, Z. Wu, T. Liu, B. Chang, X. Sun, L. Li, and Z. Sui. A survey on in-context learning, 2024. URL https://arxiv.org/abs/2301.00234

    arXiv 2024

Show all 44 references
  1. [8]

    Douze, A

    M. Douze, A. Guzhva, C. Deng, J. Johnson, G. Szilvasy, P.-E. Mazaré, M. Lomeli, L. Hosseini, and H. Jégou. The faiss library, 2024. URL https://arxiv.org/abs/2401.08281

    2024 arXiv

  2. [9]

    E. Fini, M. Shukor, X. Li, P. Dufter, M. Klein, D. Haldimann, S. Aitharaju, V . G. T. da Costa, L. Béthune, Z. Gan, et al. Multimodal autoregressive pre-training of large vision encoders. arXiv preprint arXiv:2411.14402, 2024

    2024 arXiv

  3. [10]

    Y . Gao, Y . Xiong, X. Gao, K. Jia, J. Pan, Y . Bi, Y . Dai, J. Sun, M. Wang, and H. Wang. Retrieval-augmented generation for large language mod- els: A survey, 2024. URL https://arxiv.org/abs/2312.10997

    2024 arXiv

  4. [11]

    Gekhman, G

    Z. Gekhman, G. Yona, R. Aharoni, M. Eyal, A. Feder, R. Reichart, and J. Herzig. Does fine-tuning llms on new knowledge encourage halluci- nations? arXiv preprint arXiv:2405.05904, 2024

    2024 arXiv

  5. [12]

    Gesmundo

    A. Gesmundo. A continual development methodology for large-scale multitask dynamic ml systems. arXiv preprint arXiv:2209.07326, 2022

    2022 arXiv

  6. [13]

    Z. Han, G. Zhou, R. He, J. Wang, T. Wu, Y . Yin, S. Khan, L. Yao, T. Liu, and K. Zhang. How well does gpt-4v(ision) adapt to distribu- tion shifts? a preliminary investigation, 2024. URL https://arxiv.org/ abs/2312.07424

    2024 arXiv

  7. [14]

    Helber, B

    P. Helber, B. Bischke, A. Dengel, and D. Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217–2226, 2019

    2019

  8. [15]

    Irvin, P

    J. Irvin, P. Rajpurkar, M. Ko, Y . Yu, S. Ciurea-Ilcus, C. Chute, H. Mark- lund, B. Haghgoo, R. Ball, K. Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference on artificial intelligenc...

    2019

  9. [16]

    Izacard, P

    G. Izacard, P. Lewis, M. Lomeli, L. Hosseini, F. Petroni, T. Schick, J. Dwivedi-Yu, A. Joulin, S. Riedel, and E. Grave. Atlas: Few-shot learning with retrieval augmented language models, 2022. URL https: //arxiv.org/abs/2208.03299

    2022 arXiv

  10. [18]

    K. Jin, X. Huang, J. Zhou, Y . Li, Y . Yan, Y . Sun, Q. Zhang, Y . Wang, and J. Ye. Fives: A fundus image dataset for artificial intelligence based vessel segmentation. Scientific data, 9(1):475, 2022

    2022

  11. [19]

    Kaddour, J

    J. Kaddour, J. Harris, M. Mozes, H. Bradley, R. Raileanu, and R. McHardy. Challenges and applications of large language models. arXiv preprint arXiv:2307.10169, 2023

    2023 arXiv

  12. [20]

    Z. Lan, S. Cai, X. He, and X. Wen. Fixcaps: An improved capsules network for diagnosis of skin cancer. IEEE Access, 10:76261–76267, 2022

    2022

  13. [21]

    Lewis, E

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, et al. Retrieval- augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474, 2020

    2020

  14. [22]

    H. Liu, C. Li, Y . Li, and Y . J. Lee. Improved baselines with visual in- struction tuning. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 26296–26306, 2024

    2024

  15. [23]

    N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang. Lost in the middle: How language models use long contexts,

  16. [24]

    Nguyen, T

    T. Nguyen, T. M. V o, T. V . Nguyen, H. H. Pham, and H. Q. Nguyen. Learning to diagnose common thorax diseases on chest radiographs from radiology reports in vietnamese. Plos one , 17(10):e0276545, 2022

    2022

  17. [25]

    Ortiz-Jimenez, A

    G. Ortiz-Jimenez, A. Favero, and P. Frossard. Task arithmetic in the tangent space: Improved editing of pre-trained models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  18. [26]

    O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition , pages 3498–3505. IEEE, 2012

    2012

  19. [27]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision,

  20. [28]

    Silva-Rodriguez, H

    J. Silva-Rodriguez, H. Chakor, R. Kobbi, J. Dolz, and I. B. Ayed. A foundation language-image model of the retina (flair): Encoding expert knowledge in text supervision. Medical Image Analysis , 99:103357, 2025

    2025

  21. [29]

    G. Team, P. Georgiev, V . I. Lei, R. Burnell, L. Bai, A. Gulati, G. Tanzer, D. Vincent, Z. Pan, S. Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  22. [30]

    Tschandl, C

    P. Tschandl, C. Rosendahl, and H. Kittler. The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pig- mented skin lesions. Scientific data, 5(1):1–9, 2018

    2018

  23. [31]

    D. Wang, J. Zhang, M. Xu, L. Liu, D. Wang, E. Gao, C. Han, H. Guo, B. Du, D. Tao, et al. Mtp: Advancing remote sensing foundation model via multi-task pretraining. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024

    2024

  24. [32]

    S. Wang, Y . Zhu, H. Liu, Z. Zheng, C. Chen, and J. Li. Knowledge editing for large language models: A survey. ACM Computing Surveys, 57(3):1–37, 2024

    2024

  25. [34]

    J. Wei, Y . Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yo- gatama, M. Bosma, D. Zhou, D. Metzler, et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682, 2022

    2022 arXiv

  26. [35]

    J. Wu, W. Gan, Z. Chen, S. Wan, and S. Y . Philip. Multimodal large language models: A survey. In 2023 IEEE International Conference on Big Data (BigData), pages 2247–2256. IEEE, 2023

    2023

  27. [36]

    Yang and S

    Y . Yang and S. Newsam. Bag-of-visual-words and spatial extensions for land-use classification. In Proceedings of the 18th SIGSPATIAL in- ternational conference on advances in geographic information systems, pages 270–279, 2010

    2010

  28. [37]

    S. Yin, C. Fu, S. Zhao, K. Li, X. Sun, T. Xu, and E. Chen. A survey on multimodal large language models. National Science Review, Nov

  29. [38]

    Zhang, Y

    D. Zhang, Y . Yu, J. Dong, C. Li, D. Su, C. Chu, and D. Yu. Mm-llms: Recent advances in multimodal large language models. arXiv preprint arXiv:2401.13601, 2024

    2024 arXiv

  30. [39]

    Zhang, J

    X. Zhang, J. Li, W. Chu, J. Hai, R. Xu, Y . Yang, S. Guan, J. Xu, and P. Cui. On the out-of-distribution generalization of multimodal large language models, 2024. URL https://arxiv.org/abs/2402.06599

    2024 arXiv

  31. [40]

    Zhang, K

    Y . Zhang, K. Gong, X. Ding, K. Zhang, F. Lv, K. Keutzer, and X. Yue. Towards unified and effective domain generalization. arXiv preprint arXiv:2310.10008, 2023

    2023 arXiv

  32. [41]

    doi: 10.1093/nsr/nwae403

    ISSN 2053-714X. doi: 10.1093/nsr/nwae403. URL http://dx.doi. org/10.1093/nsr/nwae403

    doi

  33. [45]

    " for demo in demo_examples: prompt += f

    W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y . Hou, Y . Min, B. Zhang, J. Zhang, Z. Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023. A Prompt used in the Visual RAG solution In this section we report the used in the proposed Visual RAG ...

    2023 arXiv

  34. [2021]

    URL https://arxiv.org/abs/2103.00020

    arXiv

  35. [2023]

    URL https://arxiv.org/abs/2307.03172

    arXiv

  36. [2024]

    URL https://arxiv.org/abs/2405.09798

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.