Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that contrasting a VLM's attention under a question with its attention under a generic description isolates the task-relevant visual signal, and that masking and magnifying only those regions improves visual question…

desk verdict Plausible training-free visual refinement with a real eval-protocol problem: hyperparameters tuned on test subsets and a definitional 'proof'. read the letter →

arxiv 2509.06461 v2 pith:LNFL2OAK submitted 2025-09-08 cs.CV cs.AI

Yuyao Ge , Shenghua Liu , Yiwei Wang , Lingrui Mei , Baolong Bi , Xuanshan Zhou , Jiayu Yao , Jiafeng Guo
show 1 more author
Xueqi Cheng
This is my paper · ORCID
classification cs.CVcs.AI
keywords vision-languagemodelsattentionentropycontrastivevisualnoisesuppressiontraining-freeenhancementquestionansweringdecompositionpixel-levelmasking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that vision-language models lose accuracy on cluttered images because visual complexity, measured by texture and color, spreads attention diffusely. It shows that attention entropy rises with complexity and falls with accuracy. The paper then proposes CARVE, a training-free method that runs the model under a question and under a generic 'describe the image' prompt, divides the question-driven attention map by the generic one pixel-wise, thresholds and crops to the top regions, and re-asks the question on the magnified crop. Across four benchmarks and four open VLMs, accuracy consistently improves, with relative gains up to 75 percent on the hardest fine-grained benchmark.

What carries the argument

The load-bearing identity is the attention decomposition $A^{(Q)}_{l,t}(I) = F_{vis}(I) \otimes F_{sem}(Q,I)$, with $\otimes$ the Hadamard product, together with the closed-form estimator $\hat{A}_i = A^{(Q)}_i / (A^{(G)}_i + \lambda)$ derived as the solution of a regularized least-squares problem. This estimator produces a pixel-level mask by fusing attention maps over a chosen layer range and generation time steps, thresholding at a top-p percentile, and cropping the top-K connected regions before resizing. The paper also introduces two complexity measures, edge density for texture and hue entropy for color, to support the motivating correlation between complexity, attention entropy, and accuracy.

What would settle it

Measure the attention map from the generic instruction on images with a single large salient object and no question. If those attention maps are not near-uniform (entropy significantly below the maximum $\log N_v$), or if replacing the generic prompt with a deliberately uniform-attention prompt changes CARVE's accuracy gains substantially, then the assumption $F_{sem}(G,I) \approx \mathbf{1}_{N_v}$ fails and the theoretical basis collapses.

Watch

Extended reading notes

Core claim

The central claim is that a VLM's attention map under a task question factorizes into an image-inherent visual-noise component and a task-driven semantic component, written as $A^{(Q)} = F_{vis}(I) \otimes F_{sem}(Q,I)$. The paper argues that a generic instruction approximates the noise term alone because its semantic factor is uniform. Consequently, the ratio $\hat{A}_i = A^{(Q)}_i / (A^{(G)}_i + \lambda)$ estimates the semantic component, because the noise divides out when it dominates $\lambda$. Using this estimated map to build a binary mask, CARVE crops the top regions and resizes them back to the original frame, which the paper shows moves correct-token probabilities ahead of incorrect ones and lifts accuracy consistently on A-OKVQA, POPE, V*, and TextVQA.

Load-bearing premise

The central premise is that a generic description prompt produces attention that captures only image-inherent visual noise, with the semantic factor essentially uniform; if the generic prompt also focuses on salient objects or carries its own bias, the division does not isolate the task-relevant signal.

Editorial extensions

If this is right

  • CARVE requires no training or external segmentation tools; it relies only on the model's own attention, so any VLM with accessible attention maps can adopt it as a drop-in inference-time step.
  • Because the generic-instruction attention depends only on the image, it can be cached and reused across many questions on the same image, lowering the per-question computational overhead.
  • The entropy-complexity correlation suggests that image difficulty can be estimated before inference, potentially guiding when to apply CARVE and how aggressive the mask should be.
  • Gains are largest for weaker models and smallest for stronger recent ones, implying visual-noise interference is a substantial bottleneck in early VLMs and partly already mitigated in newer training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the decomposition generalizes, the same contrast could be applied to prediction heads or detection outputs, turning the semantic map into a prompt-conditional prior rather than a one-shot crop.
  • The approach suggests a diagnostic tool: attention entropy or the contrast map could score how much of a VLM's error is due to visual clutter versus reasoning, supporting finer-grained failure analysis.
  • A direct stress test would be to replace the generic prompt with one that explicitly demands uniform scanning of the whole image; if CARVE's gains vanish or vary strongly, the noise-isolation assumption is fragile.
  • The same pixel-level contrast could be applied to intermediate features rather than attention maps, potentially recovering semantic signal even when attention itself is noisy.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes CARVE, a training-free method that improves VLM visual question answering by masking image regions deemed irrelevant. The method extracts attention maps under a task question and under a general instruction, computes a contrastive ratio A^(Q)/(A^(G)+lambda), thresholds and crops the resulting map, and re-runs the VLM on the refined image. The manuscript first reports correlational evidence that visual complexity (texture and color) correlates with attention entropy and that entropy correlates negatively with accuracy. It then presents a theoretical decomposition of attention into a visual-noise factor and a semantic-signal factor, claims a closed-form solution, and reports consistent accuracy gains over four datasets (A-OKVQA, POPE, V*, TextVQA) and four VLMs, with relative gains up to about 75% on V*. The paper also includes ablations over time steps, layer ranges, mask percentiles, and keep-region counts, plus a comparison with external tool-based baselines.

Significance. If the empirical claim is verified, CARVE would be a useful, simple, and inexpensive pre-processing method for improving VLM visual reasoning, and the attention-entropy analysis would contribute a genuinely interesting observation about how visual clutter degrades attention. The paper has several concrete strengths: it evaluates on four models and four datasets, includes ablations over time steps and layers, compares against SAM/YOLO/CLIP and ViCrop baselines, and gives detailed implementation information. The 'training-free' property and the pixel-level masking idea are clearly stated and easy to reproduce in principle. However, the current evaluation protocol does not yet support the 'consistently enhances' claim, and the theoretical decomposition is definitional rather than derived; these two issues are load-bearing and require revision before the central claims can be accepted.

major comments (4)
  1. [Appendix D and Figure 7] Hyperparameters are selected on evaluation data, so the reported gains are likely optimistic. Appendix D chooses the general instruction G by maximizing accuracy on a 1,000-instance TextVQA subset, and Figure 7 selects p and K on another 1,000-instance TextVQA subset; the same TextVQA set then appears in Tables 1 and 2. No separate validation split is held out before the best configuration is reported. Because several reported deltas are small in absolute terms (e.g., A-OKVQA QWEN2.5-VL-3B +4.79 to +7.26; POPE +0.23 to +1.73), selection effects alone could account for part or all of the observed improvements. Please report a validation protocol with a disjoint test set, or at minimum provide per-configuration confidence intervals on a held-out split.
  2. [Section 4.1 and Appendix C.1] The attention-decomposition 'proof' is circular. In Definition 1, Eq. (4.1) postulates A(Q) = F_vis ⊗ F_sem, and Eq. (4.2) sets A(G) ≈ F_vis by assuming F_sem(G,I) ≈ 1_Nv. Theorem C.1 then defines F_vis through A(G) and obtains the decomposition by construction; the closed-form solution in Eq. (4.6) follows algebraically from these definitions rather than from an independent argument. The assumption that general-instruction attention isolates image-inherent noise is empirically testable, and it is not tested here. Please either re-frame Section 4.1 as a heuristic motivation (which is acceptable), or provide a validation of the assumption, for example by comparing the general-instruction mask against human-labeled distractors or an independent noise estimate.
  3. [Tables 1, 2, and Appendix E] The experiments lack error bars and significance tests, and the V* benchmark contains only 191 questions, so several headline improvements correspond to very small absolute counts. For instance, a 75.24% relative improvement for LLAVA1.5-13B on V* is roughly a 32-question swing, and the POPE deltas of +0.23 to +3.10 are within typical run-to-run variation for binary probing. Greedy decoding is deterministic, but the selection of attention layers and time steps still introduces multiple-comparison risk. Please report standard errors or bootstrap confidence intervals, and state how many protocol choices were explored before the reported configuration was selected.
  4. [Eq. (4.6) and Theorem C.4] The theoretical error analysis does not repair the circularity. Theorem C.4 assumes F_sem(G,I) = 1_Nv + epsilon and then bounds the error of the estimator A^(Q)/(A^(G)+lambda) relative to F_sem; this is a consistency check of the algebraic estimator, not an independent derivation. The paper would be strengthened by explicitly acknowledging that the decomposition is an assumption and by concentrating the theoretical contribution on the convexity and stability properties of the optimization, which are proved correctly in Appendices C.2 and C.3.
minor comments (6)
  1. [Title and header] The title and running header contain 'BYCONTRASTIVEATTENTION' with missing spaces; please fix the formatting.
  2. [Eq. (3.3)] The notation a_{l,t,i} is described as the 'contrasted attention weight' but the contrast operation is not defined at this point; please specify the normalization or the contrast step used before entropy is computed.
  3. [Table 3] The ViCrop variants 'rel-att', 'grad-att', and 'pure-grad' are not defined in the main text or appendix; please add a one-sentence description of each variant so the comparison is interpretable.
  4. [Appendix C.6] Theorem C.6 invokes the Data Processing Inequality, but the Data Processing Inequality applies to Markov chains of random variables, not to arbitrary layer-wise attention maps in a transformer. The monotonic decrease of entropy with depth may hold empirically, but the proof as written is not valid; rephrase this as an empirical observation or provide a proper information-theoretic argument.
  5. [Figure 4] The correlation plots bin complexity into intervals and show the average entropy per bin, but they do not show the distribution of samples within bins or confidence intervals; adding scatter density and per-bin confidence intervals would make the claimed monotonic relationship more convincing.
  6. [Algorithm 1] The symbol M is used both for the VLM model and for the binary mask M*, which is confusing in lines 1, 3, and 7; please use distinct symbols, such as V for the model and B for the mask.

Circularity Check

3 steps flagged · score 6.0 of 10

Theoretical attention decomposition is definitional; TextVQA hyperparameters are selected on evaluation data.

  1. self definitional [Section 4.1, Definition 1 and Eq. 4.2; Appendix C.1, Theorem C.1]
    "When using general instructions G, due to the absence of specific tasks to introduce semantic information, the semantic signal function reduces to uniform distribution (F_sem(G,I)≈1_Nv), making general instruction attention predominantly capture visual noise: A^(G)_l,t(I)≈F_vis(I). ... Given the boundary condition that F_sem(G,I)=1_Nv when Q=G, we obtain: φ(F_vis(I))=φ(A^(G)_l,t(I))."

    Eq. 4.2 defines F_vis(I) to be A^(G)(I) by assumption, and Theorem C.1's boundary condition then forces F_sem(Q,I)=A^(Q)(I)/A^(G)(I). The 'existence and uniqueness' proof is therefore a definitional identity: any two positive attention arrays can be decomposed as one array times the elementwise ratio. The claimed separation of visual signal into semantic signal and visual noise is not derived from independent evidence; it is installed as the starting definition, so the later theorems cannot independently support it.

  2. self definitional [Section 4.1, Eq. 4.4-4.6; Appendix C.3, Theorem C.3]
    "Substituting Definition 1's relationships A^(Q)_i≈F_vis,i·F_sem,i and A^(G)_i≈F_vis,i into the optimization objective yields ... Solving the first-order optimality conditions yields the closed-form solution: Â_i = A^(Q)_i/(A^(G)_i+λ)."

    The objective J(Ã) is constructed from the same assumed decomposition, so minimizing it recovers the algebraic identity  = A^(Q)/(A^(G)+λ). Eq. 4.6 is a restatement of Definition 1 plus Eq. 4.2, not an independent derivation that attention contrast isolates visual noise. The calculus in Theorem C.3 is correct, but the 'semantic extraction' result is equivalent to the input assumption by construction.

1 more flagged steps
  1. fitted input called prediction [Appendix D, Table 4; Section 5.2 'Sensitivity Analysis of Mask Generation', Figure 7]
    "To identify the optimal general instruction ... we conducted experiments on a randomly sampled subset of 1000 instances from the TextVQA dataset using the QWEN2.5-VL-3B. ... We examine a 1,000-instance subset randomly sampled from TextVQA. As shown in Figure 7, when p is set within [0.2,0.6] combined with K∈{2,3}, the model achieves optimal performance."

    The general instruction and mask hyperparameters p and K are selected by maximizing accuracy on TextVQA subsets, and the same TextVQA benchmark is then reported in Tables 1-2 as improved by CARVE, with no separate held-out validation split described. The TextVQA gains are therefore partly fitted to closely related evaluation data before being presented as predictions. This does not force the empirical result mathematically, but it converts part of the reported evaluation on TextVQA into a selection artifact.

full rationale

The central theoretical claim—that contrasting question attention with general-instruction attention decomposes visual signal into semantic signal and visual noise—is true only by the paper's own definitions. Eq. 4.2 sets F_vis(I)=A^(G)(I), and Theorem C.1 uses the boundary condition F_sem(G,I)=1_Nv to define the semantic factor as the ratio A^(Q)/A^(G). The closed-form solution in Eq. 4.6 then follows from this assumption; it is a tautological restatement rather than an independently proven visual-noise-suppression result. The empirical benchmark comparisons are real experiments, but Appendix D and the Figure 7 sensitivity analysis select the general instruction and mask parameters on subsets of TextVQA, and the same benchmark is subsequently reported as the CARVE improvement without a described held-out split, so the 'consistently enhances' claim on TextVQA is partly fitted rather than predicted. Self-citations in the paper are not load-bearing: the entropy metric is credited to prior overlapping-author work, but the theoretical and empirical claims do not depend on that citation. Overall, the empirical method has independent content and is externally testable, but the theory reduces to definitions and a key evaluation is partially fitted, giving a partial-circularity score of 6.

Assumptions & free parameters 5 free parameters · 4 assumptions · 2 invented entities

The 'theoretical foundation' rests on a decomposition that is true by construction: F_vis is set equal to A(G) and F_sem to A(Q)/A(G). The optimization objective Eq 4.4 is ad hoc, chosen so that the closed form becomes the contrast ratio. Hyperparameters λ, p, K, L, and T are tuned on the evaluation benchmarks with no separate validation set.

free parameters (5)
  • λ (regularization) = not reported
    Controls the contrast-ratio denominator; no value given in the paper, presumably tuned.
  • p (retained pixel percentile) = 0.2 to 0.6 (Figure 7)
    Chosen by grid search on a 1000-instance TextVQA subset; affects amount of masking.
  • K (max keep regions) = 2 to 3 (Figure 7)
    Chosen by grid search; controls how many connected regions are preserved.
  • L (layer range) = [20,25] (Tables 1 and 2)
    Selected based on ablation; attention maps from these layers give best results.
  • T (generation time steps) = t_end (Table 1)
    Final token attention selected after comparing t_start, t_end, and T_full.
assumptions (4)
  • domain assumption General instruction attention approximates visual noise: F_sem(G,I) ≈ 1_Nv, making A(G) ≈ F_vis(I).
    Used in Eq 4.2 and Theorem C.1; unproven and may fail because general prompts still attend to salient objects.
  • ad hoc to paper Attention decomposition A(Q) = F_vis ⊗ F_sem exists uniquely.
    Theorem C.1 defines F_vis=A(G) and F_sem=A(Q)/A(G), so it is true by construction, not an empirical discovery.
  • domain assumption Canny edge density and hue entropy quantify visual complexity relevant to attention.
    Section 3.2; arbitrary choices not validated against human perception or alternative complexity measures.
  • ad hoc to paper Data Processing Inequality implies attention entropy monotonically decreases with layer depth.
    Theorem C.6 misapplies DPI to attention maps; attention entropy is not necessarily monotone across layers.
invented entities (2)
  • F_vis(I), the visual noise factor
    purpose: Represents image-inherent, question-independent attention bias used as the denominator in contrastive refinement.
    Defined from general-instruction attention; no external measurement or falsifiable prediction is provided.
  • F_sem(Q,I), the semantic signal factor
    purpose: Represents task-relevant attention to be extracted by the contrastive method.
    Defined as the ratio A(Q)/A(G), making the extraction circular with the method itself.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning." pith.science (2026). https://pith.science/paper/LNFL2OAK

@misc{pith2026250906461,
  author       = {Pith},
  title        = {Pith review of: Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LNFL2OAK}},
  note         = {Machine review of arXiv:2509.06461}
}
read the original abstract

Vision-Language Models (VLMs) have demonstrated remarkable success across diverse visual tasks, yet their performance degrades in complex visual environments. While existing enhancement approaches require additional training, rely on external segmentation tools, or operate at coarse-grained levels, they overlook the innate ability within VLMs. To bridge this gap, we investigate VLMs' attention patterns and discover that: (1) visual complexity strongly correlates with attention entropy, negatively impacting reasoning performance; (2) attention progressively refines from global scanning in shallow layers to focused convergence in deeper layers, with convergence degree determined by visual complexity. (3) Theoretically, we prove that the contrast of attention maps between general queries and task-specific queries enables the decomposition of visual signal into semantic signals and visual noise components. Building on these insights, we propose Contrastive Attention Refinement for Visual Enhancement (CARVE), a training-free method that extracts task-relevant visual signals through attention contrasting at the pixel level. Extensive experiments demonstrate that CARVE consistently enhances performance, achieving up to 75% improvement on open-source models. Our work provides critical insights into the interplay between visual complexity and attention mechanisms, offering an efficient pathway for improving visual reasoning with contrasting attention.

Figures

Figures reproduced from arXiv: 2509.06461 by the authors.

Figure 1
Figure 1. The effect of manually progressive masking on candidate tokens probabilities predicted [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Attention maps across different layers during inference. Each row represents a visual [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Visualization of texture and color complexity analysis. Each row represents a sample [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Correlation analysis between visual complexity and attention entropy. Both attention [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Attention entropy’s correlation with accuracy and its evolution across layers. Shaded [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: CARVE comprises three stages: Stage 1 generates general attention distribution [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Impact of mask generation hyperparameters on TextVQA accuracy for Q [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Images masked with CARVE. The caption of each subfigure shows the computed threshold [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: Images masked with CARVE. The caption of each subfigure shows the computed threshold [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Video LLMs understand harmful content but activate weaker refusal signals when the query is benign; prompt realignment reduces attack success from ~48% to ~1%.

Reference graph

Works this paper leans on

25 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [1]

    Star attention: Efficient llm inference over long sequences.arXiv preprint arXiv:2411.17116,

    Shantanu Acharya, Fei Jia, and Boris Ginsburg. Star attention: Efficient llm inference over long sequences.arXiv preprint arXiv:2411.17116,

    arXiv

  2. [6]

    Dp-iqa: Utilizing diffusion prior for blind image quality assessment in the wild.arXiv preprint arXiv:2405.19996,

    Honghao Fu, Yufei Wang, Wenhan Yang, Alex C Kot, and Bihan Wen. Dp-iqa: Utilizing diffusion prior for blind image quality assessment in the wild.arXiv preprint arXiv:2405.19996,

    arXiv

  3. [7]

    Vistawise: Build- ing cost-effective agent with cross-modal knowledge graph for minecraft.arXiv preprint arXiv:2508.18722,

    Honghao Fu, Junlong Ren, Qi Chai, Deheng Ye, Yujun Cai, and Hao Wang. Vistawise: Build- ing cost-effective agent with cross-modal knowledge graph for minecraft.arXiv preprint arXiv:2508.18722,

    work page arXiv

  4. [8]

    Mrfd: Multi-region fusion decoding with self-consistency for mitigating hallucinations in lvlms, 2025a

    Haonan Ge, Yiwei Wang, Ming-Hsuan Yang, and Yujun Cai. Mrfd: Multi-region fusion decoding with self-consistency for mitigating hallucinations in lvlms, 2025a. URLhttps://arxiv. org/abs/2508.10264. Yuyao Ge, Zhongguo Yang, Lizhe Chen, Yiming Wang, and Chengyang Li. Attack based on data: a novel perspective to attack sensitive points directly.Cybersecurity,...

    arXiv

  5. [10]

    Distillm-2: A contrastive approach boosts the distillation of llms.arXiv preprint arXiv:2503.07067,

    Jongwoo Ko, Tianyi Chen, Sungnyun Kim, Tianyu Ding, Luming Liang, Ilya Zharkov, and Se- Young Yun. Distillm-2: A contrastive approach boosts the distillation of llms.arXiv preprint arXiv:2503.07067,

    arXiv

  6. [11]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning, pp. 19730–19742. PMLR, 2023a. Tianhao Li, Jingyu Lu, Chuangxin Chu, Tianyu Zeng, Yujia Zheng, Mei Li, Haotian Huang, Bin Wu, Zuoxian Liu, Kai Ma, et a...

    work page 2025

  7. [12]

    Evaluating object hallucination in large vision-language models.arXiv preprint arXiv:2305.10355, 2023b

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models.arXiv preprint arXiv:2305.10355, 2023b. Zhecheng Li, Guoxian Song, Yujun Cai, Zhen Xiong, Junsong Yuan, and Yiwei Wang. Texture or semantics? vision-language models get lost in font recognition. InConference on Langu...

    arXiv 2025

  8. [13]

    Structured attention matters to multimodal llms in document understanding.arXiv preprint arXiv:2506.21600, 2025a

    Chang Liu, Hongkai Chen, Yujun Cai, Hang Wu, Qingwen Ye, Ming-Hsuan Yang, and Yiwei Wang. Structured attention matters to multimodal llms in document understanding.arXiv preprint arXiv:2506.21600, 2025a. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.arXiv preprint arXiv:2304.08485, 2023a. Haotian Liu, Chunyuan Li, Yuhe...

    arXiv

Show all 25 references
  1. [14]

    Smooth reading: Bridging the gap of recurrent llm to self-attention llm on long-context tasks.arXiv preprint arXiv:2507.19353, 2025b

    Kai Liu, Zhan Su, Peijie Dong, Fengran Mo, Jianfei Gao, ShaoTing Zhang, and Kai Chen. Smooth reading: Bridging the gap of recurrent llm to self-attention llm on long-context tasks.arXiv preprint arXiv:2507.19353, 2025b. Yixin Liu, Kejian Shi, Katherine S He, Longtian Ye, Alexa...

    arXiv

  2. [15]

    Slang: New concept comprehension of large language models.arXiv preprint arXiv:2401.12585, 2024a

    Lingrui Mei, Shenghua Liu, Yiwei Wang, Baolong Bi, and Xueqi Cheng. Slang: New concept comprehension of large language models.arXiv preprint arXiv:2401.12585, 2024a. Lingrui Mei, Shenghua Liu, Yiwei Wang, Baolong Bi, Jiayi Mao, and Xueqi Cheng. ” not aligned” is not” malicious...

    arXiv

  3. [16]

    Qwen2.5-vl: A powerful vision-language model for seamless computer interaction.arXiv preprint arXiv:2409.12191,

    Qwen. Qwen2.5-vl: A powerful vision-language model for seamless computer interaction.arXiv preprint arXiv:2409.12191,

    arXiv

  4. [18]

    Anne M Treisman and Garry Gelade

    URLhttps://arxiv.org/abs/2507.01006. Anne M Treisman and Garry Gelade. A feature-integration theory of attention.Cognitive psychol- ogy, 12(1):97–136,

    arXiv

  5. [19]

    V*: Guided visual search as a core mechanism in multimodal llms

    Penghao Wu and Saining Xie. V*: Guided visual search as a core mechanism in multimodal llms. arXiv preprint arXiv:2312.14135,

    arXiv

  6. [20]

    Butterfly effects in toolchains: A comprehensive analysis of failed parameter filling in llm tool-agent systems.arXiv preprint arXiv:2507.15296,

    Qian Xiong, Yuekai Huang, Ziyou Jiang, Zhiyuan Chang, Yujia Zheng, Tianhao Li, and Mingyang Li. Butterfly effects in toolchains: A comprehensive analysis of failed parameter filling in llm tool-agent systems.arXiv preprint arXiv:2507.15296,

    arXiv

  7. [21]

    Who is in the spotlight: The hidden bias undermining multimodal retrieval- augmented generation.arXiv preprint arXiv:2506.11063, 2025a

    Jiayu Yao, Shenghua Liu, Yiwei Wang, Lingrui Mei, Baolong Bi, Yuyao Ge, Zhecheng Li, and Xueqi Cheng. Who is in the spotlight: The hidden bias undermining multimodal retrieval- augmented generation.arXiv preprint arXiv:2506.11063, 2025a. Jiayu Yao, Shenghua Liu, Yiwei Wang, Li...

    arXiv

  8. [22]

    ISBN 979-8-89176-251-0

    Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long.1367. URLhttps://aclanthology. org/2025.acl-long.1367/. Guangzi Zhang, Lizhe Chen, Yu Zhang, Yan Liu, Yuyao Ge, and Xingquan Cai. Translating words to worlds: zero-shot synthesis ...

    2025 arXiv

  9. [23]

    Truthx: Alleviating hallucinations by editing large lan- guage models in truthful space.arXiv preprint arXiv:2402.17811, 2024b

    Shaolei Zhang, Tian Yu, and Yang Feng. Truthx: Alleviating hallucinations by editing large lan- guage models in truthful space.arXiv preprint arXiv:2402.17811, 2024b. 13 Preprint. Under review. Yujia Zheng, Tianhao Li, Haotian Huang, Tianyu Zeng, Jingyu Lu, Chuangxin Chu, Yuek...

    arXiv

  10. [24]

    Factual dialogue summarization via learning from large language models.arXiv preprint arXiv:2406.14709,

    Rongxin Zhu, Jey Han Lau, and Jianzhong Qi. Factual dialogue summarization via learning from large language models.arXiv preprint arXiv:2406.14709,

    arXiv

  11. [25]

    Write a general description of the image

    14 Preprint. Under review. A DEFINITION ANDEXPLANATION A.1 DEFINITION Symbol Definition Description I∈R H×W×3 Input image Image with heightHand widthW H,WImage dimensions Height and width in pixels QTask-specific question Task-specific question GGeneral instruction General ins...

    2025

  12. [1986]

    Frequency-importance gaussian splatting for real-time lightweight radiance field rendering.Multimedia Tools and Ap- plications, 83(35):83377–83401, 2024a

    Lizhe Chen, Yan Hu, Yu Zhang, Yuyao Ge, Haoyu Zhang, and Xingquan Cai. Frequency-importance gaussian splatting for real-time lightweight radiance field rendering.Multimedia Tools and Ap- plications, 83(35):83377–83401, 2024a. Lizhe Chen, Binjia Zhou, Yuyao Ge, Jiayi Chen, and ...

    arXiv

  13. [2016]

    A-okvqa: A benchmark for visual question answering using world knowledge

    Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. A-okvqa: A benchmark for visual question answering using world knowledge. InComputer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceed- ings, Pa...

    2022

  14. [2022]

    Context-dpo: Aligning language models for context-faithfulness.arXiv preprint arXiv:2412.15280,

    Baolong Bi, Shaohan Huang, Yiwei Wang, Tianchi Yang, Zihan Zhang, Haizhen Huang, Lingrui Mei, Junfeng Fang, Zehao Li, Furu Wei, et al. Context-dpo: Aligning language models for context-faithfulness.arXiv preprint arXiv:2412.15280,

    arXiv

  15. [2023]

    Under review

    10 Preprint. Under review. Yuyao Ge, Shenghua Liu, Baolong Bi, Yiwei Wang, Lingrui Mei, Wenjie Feng, Lizhe Chen, and Xueqi Cheng. Can graph descriptive order affect solving graph problems with llms?ACL 2025, pp. 6404–6420, 2025b. Yuyao Ge, Shenghua Liu, Yiwei Wang, Lingrui Mei...

    2025 arXiv

  16. [2024]

    Decoding by contrasting knowledge: Enhancing large language model confidence on edited facts

    Baolong Bi, Shenghua Liu, Lingrui Mei, Yiwei Wang, Junfeng Fang, Pengliang Ji, and Xueqi Cheng. Decoding by contrasting knowledge: Enhancing large language model confidence on edited facts. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pile- hvar (eds.)...

    arXiv

  17. [2025]

    Lifelong knowledge editing for LLMs with retrieval-augmented continuous prompt learn- ing

    Qizhou Chen, Taolin Zhang, Xiaofeng He, Dongyang Li, Chengyu Wang, Longtao Huang, and Hui Xue’. Lifelong knowledge editing for LLMs with retrieval-augmented continuous prompt learn- ing. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.),Proceedings of the 2024 Confer...

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.