Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels

Yao Zhang; Yu Xiao; Zhuchenyang Liu

REVIEW 2 major objections 8 minor 15 references

Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels

T0 review · 2 major / 8 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Open vision-language models already locate document evidence in language; forcing them to emit coordinates is what collapses attribution.

desk verdict Clean interface diagnosis with real recall gains; the “models already know locations in language” claim is a bit stronger than the ablations strictly force. read the letter →

arxiv 2607.24651 v1 pith:B5ZHZSXG submitted 2026-07-27 cs.CV cs.CLcs.IR

Zhuchenyang Liu , Yao Zhang , Yu Xiao This is my paper · ORCID

classification cs.CVcs.CLcs.IR

keywords visualdocumentunderstandingevidenceattributionhallucinationvision-languagemodelsquote-basedcitationmultimodalretrievalregion-label-freeRLGRPO

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Reliable answers over long visual documents need evidence attribution: the model must point to the page regions that support each claim. Current benchmarks force that step through bounding-box coordinates, and open models then show near-zero evidence recall and extreme Attribution Hallucination even when the answer itself is right. This paper holds models, pages, questions, and scoring fixed and changes only the evidence interface: the model quotes supporting text verbatim, and a layout parser plus multimodal retriever turns each quote into a page region. Across six open models, evidence recall jumps from single digits to the mid-twenties through mid-forties and the hallucination rate roughly halves, with little change in answer quality. The same quote-and-retrieve pipeline then becomes a training scaffold whose reward needs only gold answers and retrieved crops, so an 8B model can be improved without any region-level labels. The practical claim is that better attribution is available without a coordinate interface and without expensive box supervision.

What carries the argument

The language evidence interface: the model outputs only text (answer plus verbatim quotes of sentences, cells, captions, or notes); a layout parser proposes semantic blocks; a multimodal encoder embeds quote text and block crops; Hungarian assignment maps each quote to one block whose page and box become the citation. The same construction supplies a region-label-free GRPO reward gated on answer correctness times judged evidence relevance and coverage of the retrieved crops.

What would settle it

Re-run the same six backbones on the verified set with ablations that strip model-generated language from the query (question-only retrieval at matched citation budget, and coordinate boxes snapped to parser blocks): if those controls match or beat quote resolution on recall and Attribution Hallucination, the interface-artifact claim fails.

Watch

Extended reading notes

Core claim

On a verified bilingual single-document CiteVQA subset, the same open vision-language models that score at most about 8 percent necessary-evidence recall under coordinate emission reach 26 to 47 percent recall when they instead emit verbatim quotes resolved by multimodal retrieval over parser blocks, and Attribution Hallucination roughly halves, while answer quality stays largely stable. Using that pipeline as a GRPO training scaffold with a judge reward on gold answers and retrieved crops—no region labels—raises an 8B backbone’s strict attributed accuracy from 22.4 to 33.8.

Load-bearing premise

The recovery is mainly location information the model already expresses in generated language, not credit that mostly belongs to the external parser and retriever once any strong query is available.

Editorial extensions

If this is right

Benchmarks and systems that score evidence only through coordinate tokens will systematically understate open VLMs’ attribution ability on visual documents.
Quote-then-resolve becomes a deployable attribution path that reuses text-RAG citation practice across the page-image boundary without teaching coordinate emission.
Region-level evidence labels are not required to raise attributed accuracy: a judge over gold answers and retrieved crops is enough signal for GRPO-style training.
Parser-proposed blocks set a measurable ceiling; improving layout parsing or allowing block naming instead of free quotes can raise the attainable recall without changing the model.
Citation similarity after assignment is an abstention signal, so systems can trade coverage for precision instead of always returning a box.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If q+a retrieval already nearly matches quotes, hybrid systems that emit a short answer plus a few quotes may be more robust than pure quote or pure coordinate policies.
The same judge-on-crops reward could supervise models that name parser block IDs or emit coarse page-plus-block indices, bridging toward end-to-end region output without human boxes.
Caption-only figure handling implies a remaining gap for purely graphical evidence; pairing quotes with a light visual pointer for uncaptioned figures is a natural next test.
Closed models that already score well with coordinates may still benefit from language evidence at training time if region labels remain scarce for long documents.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing worth knowing is simple: on a verified CiteVQA single-doc split, open VLMs that barely hit single-digit box recall under coordinates jump to ~26–47% when they only quote text and a parser+multimodal retriever places the quote. Answer quality barely moves, AH roughly halves, and the flat IoU curves plus Docling replication make the geometric part hard to dismiss. The GRPO add-on without region labels then lifts an 8B from 22.4 to 33.8 SAA. That package is useful for anyone stuck training box heads on long PDFs.

What is actually new is not quote-then-cite (text RAG already did that) but the controlled head-to-head on visual documents with an explicit AH rate, the credit-assignment suite (snap, q, q+a, lex/BM25), and a reward that only needs gold answers plus retrieved crops. Design is careful: same backbones/inputs/scoring, six models, paired tests, separate train/eval judges, no train–eval PDF overlap, residual decomposition showing training mostly fixes “never quoted.” They also ship the reproducibility plan.

Soft spot, in proportion: Section 5.3’s own numbers. Question-only retrieval already triples coordinate recall; q+a often matches or beats quotes. Their defense—that q+a is a second reading of model language, plus 11.1 pts oracle-union complementarity—is reasonable, but it is interpretive. A gold-answer (or other-model-answer) retrieval control would separate “model-carried location” from “any fluent restatement + strong retriever.” Without it, the safer headline is “coordinates are broken for open models; quote+retrieve (or even answer-aware retrieve) fixes a lot,” not a pure capability-in-language story. Minor: GRPO is one seed/one backbone, precision/F1 drop with more citations, figures stay hard, filter shrinks the bench, judges everywhere.

For DocVQA, grounded RAG, and eval design people this is worth reading. Math is ordinary RL; data and citations look solid. I would send it to referees—ask them to tighten the credit-assignment wording and ideally add the gold-answer retrieval control—not desk-reject it. Engage; cite the interface comparison if you work on attribution.

Referee Report

2 major / 8 minor

Summary. The manuscript studies evidence attribution in visual document QA and argues that the widely reported failure of open VLMs to cite correct regions (Attribution Hallucination) is substantially an artifact of the coordinate output interface rather than a missing capability. On a verified 719-question bilingual subset of CiteVQA, the authors hold backbone, inputs, and scoring fixed and contrast the standard coordinate interface with a language interface in which the model quotes evidence verbatim and a multimodal retriever over layout-parser blocks resolves each quote to a region. Across six open VLMs, necessary-evidence recall rises from ≤8.1 to 25.9–46.9 and AH roughly halves, with answer quality largely unchanged. The same quote-and-retrieve pipeline is then used as a GRPO training scaffold whose reward (a VLM judge reading the gold answer and retrieved crops, gated on answer correctness) requires no region labels, raising an 8B backbone's strict attributed accuracy from 22.4 to 33.8. Ablations include coordinate-to-block snapping, question-only and question+answer retrieval controls, lexical/BM25 resolvers, a Docling cross-parser check, and an element-level residual error decomposition.

Significance. If the results hold, this is a useful and well-executed contribution to visual document attribution. The strengths are concrete: a controlled same-backbone/same-input/same-scoring comparison across six models and four families; judge-free geometric scoring (IoU-based recall) independent of the training reward; paired bootstrap and McNemar significance tests; a cross-parser replication (Docling, App. E) that rules out shared segmentation conventions with the annotation tooling; budget-matched retrieval controls; an honest residual decomposition (App. H, Fig. 7) showing where training gains come from; and a promised full release of the verified set, filters, and code. The region-label-free GRPO reward (Eq. 2) is a genuinely practical recipe — it uses only QA-pair data any benchmark already has — and the authors disclose its precision/F1 trade-off rather than hiding it. The interface diagnosis, even in its weaker form ("coordinates lose page-correct evidence at element granularity; quotes resolved by retrieval recover it"), reframes Attribution Hallucination in a way the growing citation-VQA literature will need to engage with.

major comments (2)

[§5.3, Table 2; §6.1–6.2; App. F, Table 11] The paper's interpretive headline — that the recovery "rests on location information that the models express in generated language" (§1, §5.3) — is supported only under a specific reading of the retrieval controls. The question-only control already reaches 22.7–29.1 recall (vs. ≤8.1 for coordinates), so the parser+retriever machinery alone triples the coordinate ceiling with no model evidence expression at all; and the q+a control exceeds the quotes row for Qwen3.5-9B (32.5 vs 25.2), Gemma-3-12B (41.8 vs 29.9), and the GRPO model (52.0 vs 51.2). The authors' defenses — reading q+a as "a second reading of the same model output" (§6.2) and the 11.1 vs 2.4 oracle-union complementarity (App. F, Table 11) — are reasonable but do not exclude the alternative that any fluent text restating the answer's content gives the retriever enough to find the block. Two concrete requests: (1) add a budget-
[§5.4, Table 1 (last row); §6.3; App. H] The headline training claim (SAA 22.4→33.8) coexists with a disclosed precision drop (37.6→26.5), a flat-to-negative box F1 (32.2→28.8), and a doubling of citations per response (2.1→4.6). The authors address reward-hacking via the strict SAA variant (Ans≥4∧Rel≥4, 19.1→27.4, McNemar p=6e-7) and the App. H budget-truncation analysis, which is good practice. However, the abstract and §5.4 present SAA as the headline number while the F1 regression appears only in §5.4/App. H; since SAA contains a recall disjunct (Rec≥0.6) that rewards exactly the broader-citation behavior that degrades F1, the claim "raising strict attributed accuracy from 22.4 to 33.8" overstates the practical improvement. Please report F1 (and citations/response) alongside SAA in Table 1's discussion and qualify the abstract accordingly. Relatedly, the GRPO result is a single seed on a single backbone (acknowledged in Lim

minor comments (8)

[Abstract] Stray typographical artifact: 'attribution"without' — also check line-wrap artifacts in the extracted prompts (App. J).
[§5.1] "Answer scores are stable across interfaces for five of six backbones" — but the exception, Qwen3-VL-8B (−6.2 Ans points), is exactly the backbone used for GRPO training. The abstract's "little change in answer quality" should be qualified, since the trained model's Ans (60.4) sits on a base that already lost 6.2 points under the language interface.
[§6.2, Table 4] Qwen3.5-9B cites in only 49.2% of responses under the language interface (Table 4, fmt). Please state explicitly how non-citing responses enter recall (presumably zero) in §4, and note that Table 1's judged-subset n (679 for this model) partially reflects this compliance failure rather than judge parse failures alone.
[Table 4] Qwen3.5-27B and Gemma-4-31B agree to one decimal at every IoU threshold in both coordinate and language rows. The footnote states they differ at the second decimal, but given how unusual this coincidence is, a sentence confirming the two runs' outputs were verified as distinct would preempt reader suspicion of a pipeline mix-up.
[§3.2, App. J.2] The language-interface prompt (J.2) requests a 0-based page index per quote, but §3.2 states page predictions are deliberately ignored. Either remove the field from the contract or add one sentence explaining why it is solicited (e.g., kept for future use / ablation); as written, the pipeline discards information it asks the model to produce.
[Fig. 5] Steps 1–44 of training metrics were lost to a cluster requeue (Fig. 5 caption). Consider noting in §A whether early-training behavior (e.g., initial quote-count drift) is observable from the validation curve or logs; the current figure starts mid-training, which slightly weakens the coverage-driven-gain narrative of §6.3.
[§4, App. H] The inherited precision metric (App. H) is nonstandard — numerator counts hit elements, denominator counts cited boxes, and it can exceed 1. This is disclosed in App. H but §4's one-line description ("follows the loop semantics") is easy to miss on first reading; add a forward pointer or one clarifying clause in §4.
[Fig. 1] Figure 1 is effective but the coordinate box example shows a case where IoU=0 despite a correct page; a sentence noting this is representative (per Fig. 3's page-vs-IoU decay) rather than cherry-picked would help.

Circularity Check

1 steps flagged · score 1.0 of 10

Empirical interface comparison and RL scaffold against external geometric benchmarks; no load-bearing circular derivation.

other [Section 4 Metrics; Eq. 2; Section 5.4 / 6.3]
"The Strict Attribution Accuracy (SAA) is defined as SAA=1[Ans≥4∧(Rel≥4∨Rec≥0.6)]. From this, we compute the Attribution Hallucination (AH) rate: AH=P(SAA=0|Ans≥4)... The reward is defined as r=a/5·(erel+ecov)/10. ... the independent judge, which reads the crops and penalizes irrelevant ones"

Not a true by-construction loop: training and evaluation use different judges and SAA also has a Rec≥0.6 geometric disjunct. Still, both the reward and Rel score a VLM reading of retrieved crops, so part of the headline SAA/AH improvement shares the same evidence channel as the training signal. Geometric Rec.5 (39.9→51.3) and cross-parser results remain independent; this is only a mild coupling, not a fitted-input-called-prediction or self-definitional collapse.

full rationale

This is an experimental systems paper, not a first-principles derivation. The central claims (coordinate vs. language interface on CiteVQA; GRPO without region labels) are tested with judge-free box IoU recall, a verified external split, paired significance tests, a cross-parser check (Docling), and a training set (LongDocURL) with no PDF overlap to evaluation. The reward (Eq. 2) is a separate VLM judge on gold answer plus retrieved crops and is distinct from the evaluation judge (Gemini-3.5-Flash vs. Qwen3.5-9B); geometric Rec.5 and multi-threshold recall do not pass through the reward. Nothing is defined as the quantity it then “predicts,” no uniqueness theorem is imported from the authors, and self-citation is not load-bearing. The only mild softness is that SAA/AH partly use a relevance judge on crops—the same modality the reward uses—so judged attributed accuracy is not fully independent of the training signal’s form; pure geometric metrics and the stricter Ans∧Rel variant still move in the same direction. Interpretive debate over whether q+a retrieval credits the model’s language vs. the retriever is a causal-attribution dispute, not circularity by construction. Score 1 reflects that minor judge-metric coupling only.

Assumptions & free parameters 5 free parameters · 6 assumptions · 3 invented entities

The central claim rests on standard VLM/RAG machinery plus several domain assumptions about evaluation filters, IoU matching, judge reliability, and parser reachability. No new physical entities; free parameters are ordinary RL/inference hyperparameters rather than fitted scientific constants. Load-bearing modeling choices are the verbatim-quote contract, whole-document assignment ignoring predicted pages, and the multiplicative reward gate.

free parameters (5)

IoU match threshold 0.5 = 0.5
Primary hit criterion inherited from CiteVQA; multi-threshold tables show language interface is flat but coordinate interface collapses—claim magnitude depends on this standard cut.
GRPO reward mix r = (a/5) * (erel + ecov)/10 = multiplicative gate; evidence sum /10
Hand-designed gating and scaling of judge subscores; shapes what 'better quoting' means and drives extra citations.
DAPO asymmetric clip (εl, εh) = (0.2, 0.28), β_KL = 0.01, n=8 rollouts = 0.2/0.28, β=0.01, n=8
RL stability hyperparameters chosen by authors; single-seed 144-step run.
Dynamic sampling keep band on mean reward = [0.01, 0.99]
Groups outside [0.01, 0.99] discarded; affects which questions contribute gradient.
Per-document pixel budget B and 1MP page cap = B=1.87e8; ≤1MP/page
Controls visual evidence available to all models; matched to CiteVQA protocol but still a design choice.

assumptions (6)

domain assumption IoU≥0.5 on the same page is the operational definition of correct element-level evidence attribution.
Section 3 problem statement and Section 4 metrics; comparability with CiteVQA requires it, but granularity mismatches inflate 'parser-unreachable' share.
domain assumption Verbatim quotes (sentence/cell/caption/note) plus crop embedding retrieval faithfully map language evidence to pixel regions, including non-text via captions.
Sections 3.1–3.2; fails for captionless pure graphics and for quotes spanning blocks.
domain assumption Vision-language judges (Gemini-3.5-Flash eval; Qwen3.5-9B reward) are reliable enough proxies for answer quality, relevance, and coverage.
SAA/AH/Rel and the entire GRPO signal depend on judge scores; authors mitigate with independent eval judge and geometric recall.
domain assumption Filtering to 719 questions with available PDFs and verifiable annotated text does not favor either interface.
Section 4 and Appendix B; filter inspects only GT, and text-layer stats are reported, but scanned-heavy collections remain untested.
ad hoc to paper Hungarian one-to-one assignment over whole-document block similarities is an appropriate resolver (ignoring model page predictions).
Eq. 1; always returns a block even for hallucinated quotes unless post-hoc similarity abstention is added.
standard math Standard transformer VLM next-token training and GRPO/DAPO policy optimization behave as in the cited RL literature.
Shao et al. 2024; Yu et al. 2025a used as given for advantages and clipped objectives.

invented entities (3)

Language (quote-and-retrieve) evidence interface for visual documents independent evidence
purpose: Replace coordinate token emission with verbatim quotes resolved to parser blocks via multimodal retrieval.
Core methodological construct; assembled from known RAG citation and layout parsing pieces but defined here as the comparative treatment.
Region-label-free GRPO reward on judged answer + retrieved crops independent evidence
purpose: Train better quoting without box or page evidence labels.
New training scaffold; falsifiable via held-out geometric recall and independent eval judge, which the paper reports.
Attribution Hallucination (AH) rate as P(SAA=0 | Ans≥4) independent evidence
purpose: Quantify correct-answer wrong-evidence failures under a strict attributed accuracy definition.
Named in CiteVQA; operationalized here with SAA including Rel/Rec disjunct—metric construct rather than physical entity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels." pith.science (2026). https://pith.science/paper/B5ZHZSXG

@misc{pith2026260724651,
  author       = {Pith},
  title        = {Pith review of: Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B5ZHZSXG}},
  note         = {Machine review of arXiv:2607.24651}
}

read the original abstract

Reliable visual document understanding requires a model to attribute each answer to the evidence regions that support it. Recent benchmarks and systems express this step through a coordinate interface: the model outputs the coordinates of bounding boxes that mark the evidence regions in the document. Under this interface, vision-language models often fail to identify the right regions even when the answer is correct, a failure known as Attribution Hallucination. We present a study that investigates whether this failure is partially limited by what the model can express through coordinates. On a verified bilingual CiteVQA subset, we compare the coordinate interface with a language interface in which the model outputs only text, quoting its evidence verbatim, and a multimodal retriever returns the location of each quote as a page region proposed by a layout parser (tables and figures are quoted through their captions or notes); the comparison is repeated over six open vision-language models. Compared with the coordinate interface, evidence recall rises from at most 8 points to between 26 and 47 and the hallucination rate roughly halves, with little change in answer quality. Building on this comparison, we use the same quote-and-retrieve pipeline as a training scaffold: because region-level evidence labels are expensive to collect for long documents, we introduce a GRPO recipe whose reward is a judge's reading of the gold answer and crops of the retrieved regions, training the model to quote better evidence without any region labels and raising an 8B backbone's strict attributed accuracy from 22.4 to 33.8. These findings indicate a practical path to improve attribution"without a coordinate interface and without costly region-level supervision.

Figures

Figures reproduced from arXiv: 2607.24651 by the authors.

**Figure 2.** Method overview. Solid lines denote our language interface: a layout parser proposes semantic blocks, a [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Recall of necessary evidence as the overlap [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Held-out validation reward during GRPO training. The final checkpoint (step 144) is evaluated without selection. 1.0 1.5 2.0 2.5 3.0 3.5 4.0 judge score (0-5) a (answer corr.) erel (relevance) ecov (coverage) 60 80 100 120 140 training step 2.5 3.0 3.5 4.0 4.5 5.0 quot…

**Figure 5.** Figure 5: Training dynamics of the reward components [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

**Figure 6.** Figure 6: Real model outputs for the same question on the same document (which programming tool was used in [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]

**Figure 7.** Figure 7: Element-level decomposition of the 1,034 [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]

**Figure 8.** Figure 8: Judge-free selective prediction over assigned [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 3 linked inside Pith

[1]

Do not select partial text from a paragraph or a single row ,→from a table, and do not select an entire page or spanning multiple tables/paragraphs

Evidence must be at the **element level**: a complete paragraph, a complete table, a complete ,→image, or a complete note. Do not select partial text from a paragraph or a single row ,→from a table, and do not select an entire page or spanning multiple tables/paragraphs. ,→Note: This is very important and will directly affect your score
[2]

For **tables and images**, if there are captions or footnotes, they need to be annotated as ** ,→separate evidence** with their own bbox, not merged into the table/image bbox
[3]

Each piece of cited evidence text should be followed by a`<bbox />`tag indicating the ,→evidence location
[4]

When an inference step relies on multiple pieces of evidence, use multiple`<bbox />`tags ,→separately
[5]

page_number

Pure reasoning/calculation steps do not need`<bbox />`. ## Annotation Format ``` <bbox page="page_number" x1="left" y1="top" x2="right" y2="bottom" /> ``` Page numbers start from 1 (note: ignore original page numbers); coordinates are relative ,→coordinates on the page image, range 0-1000. ## Examples **Question:** What is the net change in the company's ...

2021
[10]

Do not paraphrase

Each evidence quote must be a verbatim text span copied from the page (a sentence, a table row ,→/cell, a caption, or a note). Do not paraphrase
[11]

Quote the evidence at the element level: enough text to identify the region, not a single word ,→and not a whole page
[12]

answer":

If multiple regions support the answer, give multiple quotes. ## Output Format Return a single JSON object exactly in this form: {"answer": "<your full answer text>", "evidence": [{"page": p, "quote": "<verbatim text>"}, ...]} where p is the 0-based page index in the order shown. J.3 Evaluation judge prompt: answer accuracy Official CiteVQA judge prompt, ...

Show all 15 references

[13]

Ignore wording, ,→order, formatting; judge meaning

answer_correctness: does the SYSTEM ANSWER semantically match the REFERENCE answer? 5= ,→equivalent, 3=partially correct or missing detail, 0=wrong/unrelated. Ignore wording, ,→order, formatting; judge meaning
[14]

evidence_relevance: are the cited crops the correct SOURCE that supports the reference answer? ,→5=a crop clearly contains the exact fact/figure/table-cell/text the reference answer ,→comes from, 3=crops are on-topic but do not pinpoint it (related context only), 0=crops ,→are...
[15]

answer_correctness

evidence_coverage: do the cited crops TOGETHER cover ALL the information the reference answer ,→requires? For a multi-part answer (several values/items/conditions), 5=every needed piece ,→appears in some crop, 3=about half present, 1=only a small part, 0=key pieces missing or ...
[2022]

Ahmad Mohammadshirazi, Pinaki Prasad Guha Neogi, Dheeraj Kulshrestha, and Rajiv Ramnath

Teaching language models to support answers with verified quotes.ArXiv, abs/2203.11147. Ahmad Mohammadshirazi, Pinaki Prasad Guha Neogi, Dheeraj Kulshrestha, and Rajiv Ramnath. 2025. Ar- ial: An agentic framework for document vqa with precise answer localization.ArXiv, abs/251...

2025 arXiv
[2023]

InConference on Empirical Meth- ods in Natural Language Processing

Enabling large language models to generate text with citations. InConference on Empirical Meth- ods in Natural Language Processing. Gregor Geigle, Radu Timofte, and Goran Glavas
[2024]

Edward J

Does object grounding really reduce hallu- cination of large vision-language models?ArXiv, abs/2406.14492. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models.Pr...

2021 arXiv
[2026]

the resulting literature lists were processed with Software R

Hart: High-resolution annotation-free reason- ing technique through a closed-loop framework. Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chun yue Li, and Jianfeng Gao. 2023. Set-of-mark prompt- ing unleashes extraordinary visual grounding in gpt- 4v. Haoxuan You, Haotian Zha...

2023 arXiv

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.

[1] [1]

Do not select partial text from a paragraph or a single row ,→from a table, and do not select an entire page or spanning multiple tables/paragraphs

Evidence must be at the **element level**: a complete paragraph, a complete table, a complete ,→image, or a complete note. Do not select partial text from a paragraph or a single row ,→from a table, and do not select an entire page or spanning multiple tables/paragraphs. ,→Note: This is very important and will directly affect your score

[2] [2]

For **tables and images**, if there are captions or footnotes, they need to be annotated as ** ,→separate evidence** with their own bbox, not merged into the table/image bbox

[3] [3]

Each piece of cited evidence text should be followed by a`<bbox />`tag indicating the ,→evidence location

[4] [4]

When an inference step relies on multiple pieces of evidence, use multiple`<bbox />`tags ,→separately

[5] [5]

page_number

Pure reasoning/calculation steps do not need`<bbox />`. ## Annotation Format ``` <bbox page="page_number" x1="left" y1="top" x2="right" y2="bottom" /> ``` Page numbers start from 1 (note: ignore original page numbers); coordinates are relative ,→coordinates on the page image, range 0-1000. ## Examples **Question:** What is the net change in the company's ...

2021

[6] [10]

Do not paraphrase

Each evidence quote must be a verbatim text span copied from the page (a sentence, a table row ,→/cell, a caption, or a note). Do not paraphrase

[7] [11]

Quote the evidence at the element level: enough text to identify the region, not a single word ,→and not a whole page

[8] [12]

answer":

If multiple regions support the answer, give multiple quotes. ## Output Format Return a single JSON object exactly in this form: {"answer": "<your full answer text>", "evidence": [{"page": p, "quote": "<verbatim text>"}, ...]} where p is the 0-based page index in the order shown. J.3 Evaluation judge prompt: answer accuracy Official CiteVQA judge prompt, ...