Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An automated multi-agent pipeline can turn research papers into posters approaching human quality.

desk verdict A genuinely useful dataset and benchmark for paper-to-poster generation, with an evaluation section that needs a serious revision before the quality claims are trustworthy. read the letter →

arxiv 2505.17104 v1 pith:V5MZDX22 submitted 2025-05-21 cs.CL cs.MM

Tao Sun , Enhao Pan , Zhengkai Yang , Kaixin Sui , Jiajun Shi , Xianfu Cheng , Tongliang Li , Wenhao Huang
show 3 more authors
Ge Zhang Jian Yang Zhoujun Li
This is my paper · ORCID
classification cs.CLcs.MM
keywords paper-to-postergenerationmulti-agentLLMacademicpostersLLM-as-a-JudgeP2PInstructdatasetP2PEvalbenchmarkHTMLrenderingfine-grainedevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that an LLM-based multi-agent framework can turn a research paper directly into a finished, HTML-rendered academic poster, and that the result approaches or sometimes beats posters made by the paper's own authors. The authors argue that splitting the work into three specialist agents—visual-element processing, content generation, and final assembly—each with a checker that triggers iterative revision, is what makes automatic poster generation practical. To support training and evaluation, they release a 30,000-plus instruction dataset and a benchmark of 121 paper–poster pairs with two LLM-as-judge scoring methods. If the claim holds, it means a major time-consuming part of scholarly communication can be automated, and it gives the community a standardized way to measure progress in that task.

What carries the argument

The load-bearing mechanism is the three-agent pipeline with per-agent checker modules: the Figure Agent, Section Agent, and Orchestrate Agent each produce an intermediate artifact and then verify it, looping back through 'reflection' when the checker flags a problem. The Section Agent's use of Markdown-style figure references and the Orchestrate Agent's HTML/CSS rendering with content-structure decoupling are what let the system integrate visuals and text coherently. The paper also treats the P2PEval benchmark as a key device: a Universal score (ten 0–5 criteria calibrated to human ratings through an XGBoost model, reported R²=0.92) and a FineGrain score (normalized checklist adherence computed by GPT-4o against 1,738 human-annotated items) are the instruments that convert 'looks good' into a measurable claim.

What would settle it

Have a set of fresh human annotators who were not involved in the benchmark rate a sample of P2P-generated and author-created posters side by side without knowing which is which; if their strict preferences substantially disagree with the order predicted by the Universal or FineGrain scores, the paper's central quality claim falls.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that academic poster generation is an automatable document-design task if it is decomposed into three cooperating agents whose outputs are repeatedly checked and revised. The Figure Agent extracts figures and tables with DocLayout-YOLO and generates semantic descriptions, the Section Agent turns the paper into structured poster text with references to those visuals, and the Orchestrate Agent renders the result as HTML/CSS with layout rules; each agent has a dedicated checker and reflection loop. The authors report that the full P2P system, especially with reasoning-capable LLMs such as Claude-3.7-Sonnet, receives FineGrain and Universal judge scores that approach or exceed the original author-created posters, and that the release of P2PInstruct and P2PEval gives the field both a training resource and a benchmark with human-annotated checklists.

Load-bearing premise

The claim that P2P approaches human-quality posters rests on GPT-4o-as-judge scores being reliable proxies for what human viewers actually prefer; the XGBoost calibration was trained only on original posters and P2P outputs, so a judge bias toward LLM-generated text or HTML layouts would invalidate the comparison.

Editorial extensions

If this is right

  • If the central claim is right, generating a conference-ready poster can become a one-click step from a paper's PDF, saving researchers hours of layout work.
  • The P2PInstruct dataset implies that smaller open models can be fine-tuned to beat their base versions on poster generation, and the paper shows Qwen3-P2P-8B achieving the highest ROUGE scores among all tested models.
  • The benchmark gives a standardized yardstick, so future poster generators can be compared on the same 121 paper–poster pairs rather than on ad hoc examples.
  • The paper's format comparison suggests HTML is a better output medium than SVG or LaTeX for automatic poster generation, which points toward web-native academic posters as the practical choice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that the same three-agent decomposition—extract visual elements, draft content, assemble layout—could transfer to other document-to-visual generation tasks, such as slides or infographics, where the checker-and-reflection loop would serve the same quality-control role.
  • A testable extension of the benchmark is to use the FineGrain checklist scoring beyond posters: if the normalized checklist score reflects human domain priorities here, it may generalize to faithfulness evaluation in other paper-to-visual translations, and that hypothesis can be checked directly.
  • The reported Universal-score calibration (R²=0.92) was trained on original posters and P2P outputs only; a readier test the authors leave open is whether that R² holds on posters from unrelated generator systems, which would show whether the judge calibration overfits to the paper's own pipeline.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents P2P, a multi-agent LLM-based framework that generates academic posters as HTML from research papers, along with P2PInstruct, a 30k-example instruction dataset, and P2PEval, a benchmark of 121 paper-poster pairs with dual (Universal and Fine-Grained) LLM-based evaluation methodologies. The authors evaluate 33 models, report that P2P with reasoning-enhanced LLMs approaches or sometimes exceeds human-created posters, and include ablation studies, format comparisons, and human pairwise preference results.

Significance. If the evaluation is sound, this is a useful contribution: a practical poster-generation system, the first large-scale instruction dataset for this task, a benchmark with human-annotated checklists, and open-source fine-tuned models. The paper also provides machine-checkable artifacts (released code and dataset) and a serious attempt at structured evaluation with XGBoost calibration and fine-grained checklists. The central quality claim, however, rests on LLM-as-judge scores whose validity for out-of-distribution generated posters is not established by the reported validations.

major comments (3)
  1. [§3.2.1 and Table 1] The Universal score is calibrated with XGBoost (reported R²=0.92) on 1,701 human ratings, but the text states these ratings cover only original posters and P2P outputs, excluding other multi-agent approaches. The resulting model is then applied to all 33 systems in Table 1, including direct MLLM outputs, text-only LLMs, and poster images from Tencent YuanBao. Ten-fold cross-validation on that restricted set does not test generalization across different generation pipelines, so the Universal ranking in Table 1 is not human-validated for most compared models. Please either calibrate/validate on a sample that spans all model families represented in Table 1, or explicitly restrict the Universal-score-based claims to the original-vs-P2P comparison.
  2. [§3.2.2 and Fig. 3] The FineGrain score is computed by GPT-4o directly, using human-annotated checklists from official posters, but no human scoring of generated posters against those checklists is reported. The formula S_fine = (Σs_i / ΣM_i) × 100 is only as reliable as the per-item scores s_i, and with no agreement data (e.g., correlation or item-level accuracy against human judges on generated output), the FineGrain values in Table 1 cannot support the conclusion that P2P 'approaches or sometimes exceeds the quality of human-created examples.' Please conduct a human-judge validation on a subset of generated posters (from at least the top-performing systems per Table 1) and report the agreement with FineGrain.
  3. [§4.3, Table 2, and §6] The pairwise human preference study is limited to P2P vs. YuanBao, P2P vs. Original, and YuanBao vs. Original. This is valuable evidence for those specific comparisons, but it does not justify the general claim in the conclusion that P2P 'produces posters that approach or sometimes exceed the quality of human-created examples, particularly when employing reasoning-enhanced LLMs,' because the broader 33-model comparisons in Table 1 rely on the unvalidated LLM scores discussed above. Please either add human pairwise evaluations for the models whose high LLM-based scores underpin the conclusion, or soften the conclusion to state that the quality claim is based on the limited human study plus LLM-based metrics without implying full human validation across models.
minor comments (5)
  1. [§3.2.1] The sentence 'We utilize both powerful models like GPT-4o and lighter models such as Qwen-VL-2.5-32B, ensuring the trained annotators are exposed to diverse samples to enhance generalizability' is confusing: it seems to mix the LLMs used as judges with human annotator training. Please rephrase to clarify which models are used for evaluation and which population the human annotators rated.
  2. [Table 1] The footnotes defining 'R' and 'T' are ambiguous; for example, 'Deepseek-R1RT 37/671B' and 'Qwen3RT 3/30B' are hard to parse. Please state explicitly for each row whether the output is from a reasoning/thinking mode, whether the model is text-only with Claude-provided figure descriptions, and what the slash-separated sizes mean.
  3. [§4.2] The 'Judge' metric is described as 'how frequently VLLM-based automated evaluators prefer P2P's posters over original author-created versions,' but in Table 1 a Judge value is reported for every model, presumably as that model's generated posters versus originals. Please clarify the definition and specify the VLLM used, the prompt, and the number of evaluation samples.
  4. [§3.2.1] No inter-annotator agreement is reported for the human overall scores (0–50) used to train XGBoost. Reporting agreement (e.g., Krippendorff's alpha or pairwise correlation) would strengthen the claim that the calibration target is reliable.
  5. [Appendix A / Fig. 6] Figure 6 is presented at very low resolution, making the three poster examples difficult to compare. Please provide higher-resolution images or zoom-ins for the key comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark checklists and pairwise human preference data are external to the P2P generation pipeline, and the self-referential P2PInstruct data is used only for training, not for evaluation.

full rationale

The paper's generation-to-evaluation chain is not circular. P2PInstruct is produced by the P2P pipeline and used to fine-tune Qwen3-P2P, but the P2PEval benchmark is constructed independently from 121 official paper-poster pairs (ACL 2022-2024 and SciPostLayout) with four human annotators producing 1,738 checklist items. The Fine-Grained score measures generated posters against those human-annotated official-poster checklists via S_fine = sum(s_i)/sum(M_i) x 100, which is a normalization of LLM item scores against human-derived maxima, not an identity with any fitted parameter. The Universal score is an XGBoost regression (R^2=0.92, 10-fold CV) trained on 1,701 human ratings covering original posters and P2P outputs; although P2P outputs appear in the calibration set, the score is a held-out prediction from human labels rather than a quantity forced by construction, and the strongest human-quality claim is additionally supported by direct pairwise human preference results in Table 2, which are independent of P2PEval. Self-citations in related work and prior author papers are not load-bearing; no uniqueness theorem, ansatz, or known result is smuggled in via citation. Therefore no specific circular step can be exhibited under the required standard.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new physical or conceptual entities are introduced. The free parameters listed are the calibration and engineering knobs the evaluation depends on.

free parameters (2)
  • XGBoost calibration model parameters (200 trees, tree depth, learning rate, etc.) = not fully reported; claims R2=0.92 with 10-fold CV
    The Universal Score is the output of this fitted regressor mapping the ten LLM-judged criteria (U1-U10) to human ratings; the score used in Table 1 is therefore a model prediction, not a direct measurement.
  • Figure Agent confidence threshold = not specified; starts at an initial value and is incrementally lowered
    The Figure Extractor uses a confidence threshold to filter detected visual elements; the threshold is adjusted during inference, which affects which figures are included in the poster and therefore the evaluation.
assumptions (3)
  • domain assumption GPT-4o's judgments of poster quality on the universal criteria and fine-grained checklists correlate with human judgments across all evaluated models.
    The main evaluation pipeline (Universal and FineGrain scores) is computed by GPT-4o; the only human calibration is via XGBoost on a limited subset, so the framework assumes this generalizes.
  • domain assumption Official author-created posters are a valid gold standard for academic poster quality.
    P2PEval constructs checklists from official posters and scores generated posters by fidelity to those checklists; this presumes the official posters are the correct target.
  • domain assumption HTML/CSS is a suitable and sufficient format for academic posters.
    P2P outputs HTML, and the ablation shows HTML beats SVG and LaTeX, but the benchmark itself is based on HTML-rendered posters; the assumption is that this format captures poster quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark." pith.science (2026). https://pith.science/paper/V5MZDX22

@misc{pith2026250517104,
  author       = {Pith},
  title        = {Pith review of: P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V5MZDX22}},
  note         = {Machine review of arXiv:2505.17104}
}
read the original abstract

Academic posters are vital for scholarly communication, yet their manual creation is time-consuming. However, automated academic poster generation faces significant challenges in preserving intricate scientific details and achieving effective visual-textual integration. Existing approaches often struggle with semantic richness and structural nuances, and lack standardized benchmarks for evaluating generated academic posters comprehensively. To address these limitations, we introduce P2P, the first flexible, LLM-based multi-agent framework that generates high-quality, HTML-rendered academic posters directly from research papers, demonstrating strong potential for practical applications. P2P employs three specialized agents-for visual element processing, content generation, and final poster assembly-each integrated with dedicated checker modules to enable iterative refinement and ensure output quality. To foster advancements and rigorous evaluation in this domain, we construct and release P2PInstruct, the first large-scale instruction dataset comprising over 30,000 high-quality examples tailored for the academic paper-to-poster generation task. Furthermore, we establish P2PEval, a comprehensive benchmark featuring 121 paper-poster pairs and a dual evaluation methodology (Universal and Fine-Grained) that leverages LLM-as-a-Judge and detailed, human-annotated checklists. Our contributions aim to streamline research dissemination and provide the community with robust tools for developing and evaluating next-generation poster generation systems.

Figures

Figures reproduced from arXiv: 2505.17104 by the authors.

Figure 1
Figure 1. The multi-agent architecture of P2P: papers are processed by the Figure Agent for [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. An example of the paper-to-poster transformation achieved by P2P, showing direct [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Overview of the poster evaluation framework used in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Distribution of P2PEVA L. both PDF and PNG formats. This dual-format preservation enables comprehensive evaluation of both textual content and visual layout while maintaining high-quality vector graphics and text information. As shown in the [PITH_FULL_IMAGE:figures/f…
Figure 5
Figure 5. Figure 5: The poster for this paper, powered by P2P. [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Examples of academic poster design for [ [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OmniPresent: Generating Coherent Presentation Suites from Scientific Papers

    cs.SE 2026-07 conditional novelty 6.5 of 10

    A multi-agent HTML pipeline with shared knowledge and cross-artifact verify-and-repair generates coherent poster/slides/video/page suites from papers and beats specialized baselines on OmniPreBench.

  2. PosterForest: Hierarchical Multi-Agent Collaboration for Scientific Poster Generation

    cs.AI 2025-08 unverdicted novelty 6.0 of 10

    PosterForest uses a Poster Tree intermediate representation and hierarchical multi-agent reasoning to generate coherent scientific posters without training, outperforming prior methods in evaluations.

Reference graph

Works this paper leans on

74 extracted references · 32 canonical work pages · cited by 2 Pith papers

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    The claude 3 model family: Opus, sonnet, haiku

    Anthropic. The claude 3 model family: Opus, sonnet, haiku. 2024. Available at: https: //www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Mo del_Card_Claude_3.pdf

    work page 2024

  3. [3]

    Vqa: Visual question answering

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. InProceedings of the IEEE international conference on computer vision, pages 2425–2433, 2015

    2015

  4. [4]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  5. [5]

    Touchstone: Evaluating vision-language models by language models.arXiv preprint arXiv:2308.16890, 2023

    Shuai Bai, Shusheng Yang, Jinze Bai, Peng Wang, Xingxuan Zhang, Junyang Lin, Xinggang Wang, Chang Zhou, and Jingren Zhou. Touchstone: Evaluating vision-language models by language models.arXiv preprint arXiv:2308.16890, 2023

    arXiv 2023

  6. [6]

    Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020

    work page 1901

  7. [7]

    xcot: Cross-lingual instruction tuning for cross-lingual chain-of-thought reasoning

    Linzheng Chai, Jian Yang, Tao Sun, Hongcheng Guo, Jiaheng Liu, Bing Wang, Xinnian Liang, Jiaqi Bai, Tongliang Li, Qiyao Peng, et al. xcot: Cross-lingual instruction tuning for cross-lingual chain-of-thought reasoning. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 23550–23558, 2025. 13

    work page 2025

  8. [8]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24185–24198, 2024

    2024

Show all 74 references
  1. [9]

    Xformparser: A simple and effective multimodal multilingual semi-structured form parser.arXiv preprint arXiv:2405.17336, 2024

    Xianfu Cheng, Hang Zhang, Jian Yang, Xiang Li, Weixiao Zhou, Kui Wu, Fei Liu, Wei Zhang, Tao Sun, Tongliang Li, et al. Xformparser: A simple and effective multimodal multilingual semi-structured form parser.arXiv preprint arXiv:2405.17336, 2024

    2024 arXiv

  2. [10]

    Sviptr: Fast and efficient scene text recognition with vision permutable extractor

    Xianfu Cheng, Weixiao Zhou, Xiang Li, Jian Yang, Hang Zhang, Tao Sun, Wei Zhang, Yuying Mai, Tongliang Li, Xiaoming Chen, et al. Sviptr: Fast and efficient scene text recognition with vision permutable extractor. InProceedings of the 33rd ACM International Conference on Inform...

    2024

  3. [11]

    A survey of chain of thought reasoning: advances, frontiers and future

    Z Chu et al. A survey of chain of thought reasoning: advances, frontiers and future. comput. res. repository (corr)(2023)

    2023

  4. [12]

    Mobile-bench: An evaluation benchmark for llm-based mobile agents.arXiv preprint arXiv:2407.00993, 2024

    Shihan Deng, Weikai Xu, Hongda Sun, Wei Liu, Tao Tan, Jianfeng Liu, Ang Li, Jian Luan, Bin Wang, Rui Yan, et al. Mobile-bench: An evaluation benchmark for llm-based mobile agents.arXiv preprint arXiv:2407.00993, 2024

    2024 arXiv

  5. [13]

    Frontend diffusion: Empowering self-representation of junior researchers and designers through agentic workflows.arXiv preprint arXiv:2502.03788, 2025

    Zijian Ding, Qinshi Zhang, Mohan Chi, and Ziyi Wang. Frontend diffusion: Empowering self-representation of junior researchers and designers through agentic workflows.arXiv preprint arXiv:2502.03788, 2025

    2025 arXiv

  6. [14]

    Webcode2m: A real-world dataset for code generation from webpage designs

    Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Bohua Chen, Yi Su, Dongping Chen, Siyuan Wu, Xing Zhou, et al. Webcode2m: A real-world dataset for code generation from webpage designs. InProceedings of the ACM on Web Conference 2025, pages 1834–1845, 2025

    2025

  7. [15]

    Uicopilot: Automating ui synthesis via hierarchical code generation from webpage designs

    Yi Gui, Yao Wan, Zhen Li, Zhongyi Zhang, Dongping Chen, Hongyu Zhang, Yi Su, Bohua Chen, Xing Zhou, Wenbin Jiang, et al. Uicopilot: Automating ui synthesis via hierarchical code generation from webpage designs. InProceedings of the ACM on Web Conference 2025, pages 1846–1855, 2025

    2025

  8. [16]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  9. [17]

    Llm multi-agent systems: Challenges and open problems.arXiv preprint arXiv:2402.03578, 2024

    Shanshan Han, Qifan Zhang, Yuhang Yao, Weizhao Jin, Zhaozhuo Xu, and Chaoyang He. Llm multi-agent systems: Challenges and open problems.arXiv preprint arXiv:2402.03578, 2024

    2024 arXiv

  10. [18]

    Layoutlmv3: Pre-training for document ai with unified text and image masking

    Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. Layoutlmv3: Pre-training for document ai with unified text and image masking. InProceedings of the 30th ACM International Conference on Multimedia, pages 4083–4091, 2022

    2022

  11. [19]

    Mapcoder: Multi- agent code generation for competitive problem solving.arXiv preprint arXiv:2405.11403, 2024

    Md Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. Mapcoder: Multi- agent code generation for competitive problem solving.arXiv preprint arXiv:2405.11403, 2024

    2024 arXiv

  12. [20]

    Postdoc: Generating poster from a long multimodal document using deep submodular optimization.arXiv preprint arXiv:2405.20213, 2024

    Vijay Jaisankar, Sambaran Bandyopadhyay, Kalp Vyas, Varre Chaitanya, and Shwetha Somasundaram. Postdoc: Generating poster from a long multimodal document using deep submodular optimization.arXiv preprint arXiv:2405.20213, 2024. 14

    2024 arXiv

  13. [21]

    Unlocking the conversion of web screenshots into html code with the websight dataset.arXiv preprint arXiv:2403.09029, 2024

    Hugo Laurençon, Léo Tronchon, and Victor Sanh. Unlocking the conversion of web screenshots into html code with the websight dataset.arXiv preprint arXiv:2403.09029, 2024

    2024 arXiv

  14. [22]

    Generative judge for evaluating alignment.arXiv preprint arXiv:2310.05470, 2023

    Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, and Pengfei Liu. Generative judge for evaluating alignment.arXiv preprint arXiv:2310.05470, 2023

    2023 arXiv

  15. [23]

    Docbank: A benchmark dataset for document layout analysis.arXiv preprint arXiv:2006.01038, 2020

    Minghao Li, Yiheng Xu, Lei Cui, Shaohan Huang, Furu Wei, Zhoujun Li, and Ming Zhou. Docbank: A benchmark dataset for document layout analysis.arXiv preprint arXiv:2006.01038, 2020

    2006 arXiv

  16. [24]

    Sketch2code: Evaluating vision-language models for interactive web design prototyping.arXiv preprint arXiv:2410.16232, 2024

    Ryan Li, Yanzhe Zhang, and Diyi Yang. Sketch2code: Evaluating vision-language models for interactive web design prototyping.arXiv preprint arXiv:2410.16232, 2024

    2024 arXiv

  17. [25]

    Alpacaeval: An automatic evaluator of instruction-following models, 2023

    Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models, 2023

    2023

  18. [26]

    Evaluating object hallucination in large vision-language models.arXiv preprint arXiv:2305.10355, 2023

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models.arXiv preprint arXiv:2305.10355, 2023

    2023 arXiv

  19. [27]

    Waffle: Multi-modal model for automated front-end development.arXiv preprint arXiv:2410.18362, 2024

    Shanchao Liang, Nan Jiang, Shangshu Qian, and Lin Tan. Waffle: Multi-modal model for automated front-end development.arXiv preprint arXiv:2410.18362, 2024

    2024

  20. [28]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. InText summa- rization branches out, pages 74–81, 2004

    2004

  21. [29]

    Layoutprompter: Awaken the design ability of large language models.Advances in Neural Information Processing Systems, 36, 2024

    Jiawei Lin, Jiaqi Guo, Shizhao Sun, Zijiang Yang, Jian-Guang Lou, and Dongmei Zhang. Layoutprompter: Awaken the design ability of large language models.Advances in Neural Information Processing Systems, 36, 2024

    2024

  22. [30]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  23. [31]

    Fuxiao Liu, Tianrui Guan, Zongxia Li, Lichang Chen, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. Hallusionbench: You see what you think? or you think what you see? an image-context reasoning benchmark challenging for gpt-4v (ision), llava-1.5, and other multi-modality models...

    arXiv

  24. [32]

    Aligning large multi-modal model with robust instruction tuning.CoRR, 2023

    Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Aligning large multi-modal model with robust instruction tuning.CoRR, 2023

    2023

  25. [33]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  26. [34]

    Roleagent: Building, interacting, and benchmarking high-quality role-playing agents from scripts.Advances in Neural Information Processing Systems, 37:49403–49428, 2025

    Jiaheng Liu, Zehao Ni, Haoran Que, Tao Sun, Noah Wang, Jian Yang, Hongcheng Guo, Zhongyuan Peng, Ge Zhang, Jiayi Tian, et al. Roleagent: Building, interacting, and benchmarking high-quality role-playing agents from scripts.Advances in Neural Information Processing Systems, 37:...

    2025

  27. [35]

    Lost in the middle: How language models use long contexts

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03172, 2023. 15

    2023 arXiv

  28. [36]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  29. [37]

    Textrank: Bringing order into text

    Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. InProceedings of the 2004 conference on empirical methods in natural language processing, pages 404–411, 2004

    2004

  30. [38]

    Tailored summary for automatic poster generator

    Kanya Paramita and Masayu Leylia Khodra. Tailored summary for automatic poster generator. In2016 International Conference On Advanced Informatics: Concepts, Theory And Application (ICAICTA), pages 1–6. IEEE, 2016

    2016

  31. [39]

    Learning to generate posters of scientific papers by probabilistic graphical models.Journal of Computer Science and Technology, 34:155–169, 2019

    Yu-Ting Qiang, Yan-Wei Fu, Xiao Yu, Yan-Wen Guo, Zhi-Hua Zhou, and Leonid Sigal. Learning to generate posters of scientific papers by probabilistic graphical models.Journal of Computer Science and Technology, 34:155–169, 2019

    2019

  32. [40]

    Learning to generate posters of scientific papers

    Yuting Qiang, Yanwei Fu, Yanwen Guo, Zhi-Hua Zhou, and Leonid Sigal. Learning to generate posters of scientific papers. InProceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016

    2016

  33. [41]

    Postersum: A multimodal benchmark for scientific poster summarization.arXiv preprint arXiv:2502.17540, 2025

    Rohit Saxena, Pasquale Minervini, and Frank Keller. Postersum: A multimodal benchmark for scientific poster summarization.arXiv preprint arXiv:2502.17540, 2025

    2025 arXiv

  34. [42]

    Seed-thinking-v1

    ByteDance Seed, Yufeng Yuan, Yu Yue, Mingxuan Wang, Xiaochen Zuo, Jiaze Chen, Lin Yan, Wenyuan Xu, Chi Zhang, Xin Liu, et al. Seed-thinking-v1. 5: Advancing superb reasoning models with reinforcement learning.arXiv preprint arXiv:2504.13914, 2025

    2025

  35. [43]

    De- sign2code: Benchmarking multimodal code generation for automated front-end engineer- ing.arXiv preprint arXiv:2403.03163, 2024

    Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. De- sign2code: Benchmarking multimodal code generation for automated front-end engineer- ing.arXiv preprint arXiv:2403.03163, 2024

    2024 arXiv

  36. [44]

    Unicoder: Scaling code large language model via universal code.arXiv preprint arXiv:2406.16441, 2024

    Tao Sun, Linzheng Chai, Jian Yang, Yuwei Yin, Hongcheng Guo, Jiaheng Liu, Bing Wang, Liqun Yang, and Zhoujun Li. Unicoder: Scaling code large language model via universal code.arXiv preprint arXiv:2406.16441, 2024

    2024 arXiv

  37. [45]

    Bitsai-cr: Automated code review via llm in practice.arXiv preprint arXiv:2501.15134, 2025

    Tao Sun, Jian Xu, Yuanpeng Li, Zhao Yan, Ge Zhang, Lintao Xie, Lu Geng, Zheng Wang, Yueyan Chen, Qin Lin, et al. Bitsai-cr: Automated code review via llm in practice.arXiv preprint arXiv:2501.15134, 2025

    2025 arXiv

  38. [46]

    Repofixeval: A repository-level program repair benchmark from issue discovering to bug fixing

    Tao Sun, Yang Yang, Xianfu Cheng, Jian Yang, Yintong Huo, Zhuoren Ye, Rubing Yang, Xiangyuan Guan, Wei Zhang, Hangyuan Ji, et al. Repofixeval: A repository-level program repair benchmark from issue discovering to bug fixing

  39. [47]

    Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025

    2025 arXiv

  40. [48]

    Mrweb: An exploration of generating multi-page resource-aware web code from ui designs

    Yuxuan Wan, Yi Dong, Jingyu Xiao, Yintong Huo, Wenxuan Wang, and Michael R Lyu. Mrweb: An exploration of generating multi-page resource-aware web code from ui designs. arXiv preprint arXiv:2412.15310, 2024

    2024 arXiv

  41. [49]

    Automatically generating ui code from screenshot: A divide-and-conquer- based approach.arXiv preprint arXiv:2406.16386, 2024

    Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, and Michael R Lyu. Automatically generating ui code from screenshot: A divide-and-conquer- based approach.arXiv preprint arXiv:2406.16386, 2024. 16

    2024 arXiv

  42. [50]

    Scipostlayout: A dataset for layout analysis and layout generation of scientific posters

    Hao Wang, Shohei Tanaka, and Yoshitaka Ushiku. Scipostlayout: A dataset for layout analysis and layout generation of scientific posters. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8136–8141, 2024

    2024

  43. [51]

    Multimodal query suggestion with multi-agent reinforcement learning from human feedback

    Zheng Wang, Bingzheng Gan, and Wei Shi. Multimodal query suggestion with multi-agent reinforcement learning from human feedback. InProceedings of the ACM Web Conference 2024, pages 1374–1385, 2024

    2024

  44. [52]

    Finetuned language models are zero-shot learners

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021

    2021 arXiv

  45. [53]

    Interac- tion2code: How far are we from automatic interactive webpage generation?arXiv preprint arXiv:2411.03292, 2024

    Jingyu Xiao, Yuxuan Wan, Yintong Huo, Zhiyao Xu, and Michael R Lyu. Interac- tion2code: How far are we from automatic interactive webpage generation?arXiv preprint arXiv:2411.03292, 2024

    2024

  46. [54]

    Prototype2code: End-to-end front-end code generation from ui design prototypes

    Shuhong Xiao, Yunnong Chen, Jiazhi Li, Liuqing Chen, Lingyun Sun, and Tingting Zhou. Prototype2code: End-to-end front-end code generation from ui design prototypes. InInter- national Design Engineering Technical Conferences and Computers and Information in Engineering Conferen...

    2024

  47. [55]

    Neural content extraction for poster generation of scientific papers.arXiv preprint arXiv:2112.08550, 2021

    Sheng Xu and Xiaojun Wan. Neural content extraction for poster generation of scientific papers.arXiv preprint arXiv:2112.08550, 2021

    2021 arXiv

  48. [56]

    Multilin- gual machine translation systems from microsoft for wmt21 shared task.arXiv preprint arXiv:2111.02086, 2021

    Jian Yang, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, et al. Multilin- gual machine translation systems from microsoft for wmt21 shared task.arXiv preprint arXiv:2111.02086, 2021

    2021 arXiv

  49. [57]

    Evaluating and aligning codellms on human preference

    Jian Yang, Jiaxi Yang, Ke Jin, Yibo Miao, Lei Zhang, Liqun Yang, Zeyu Cui, Yichang Zhang, Binyuan Hui, and Junyang Lin. Evaluating and aligning codellms on human preference. arXiv preprint arXiv:2412.05210, 2024

    2024 arXiv

  50. [58]

    Execrepobench: Multi-level executable code completion evaluation.arXiv preprint arXiv:2412.11990, 2024

    Jian Yang, Jiajun Zhang, Jiaxi Yang, Ke Jin, Lei Zhang, Qiyao Peng, Ken Deng, Yibo Miao, Tianyu Liu, Zeyu Cui, et al. Execrepobench: Multi-level executable code completion evaluation.arXiv preprint arXiv:2412.11990, 2024

    2024 arXiv

  51. [59]

    Multi-agent collaboration for multilingual code instruction tuning.arXiv preprint arXiv:2502.07487, 2025

    Jian Yang, Wei Zhang, Jiaxi Yang, Yibo Miao, Shanghaoran Quan, Zhenhe Wu, Qiyao Peng, Liqun Yang, Tianyu Liu, Zeyu Cui, et al. Multi-agent collaboration for multilingual code instruction tuning.arXiv preprint arXiv:2502.07487, 2025

    2025 arXiv

  52. [60]

    Scipg: A new benchmark and approach for layout-aware scientific poster generation

    Kaichun Yao, Lan Zeng, Chuan Qin, Hengshu Zhu, Yanjun Wu, and Libo Zhang. Scipg: A new benchmark and approach for layout-aware scientific poster generation

  53. [61]

    Web2code: A large-scale webpage-to-code dataset and evaluation framework for multimodal llms.arXiv preprint arXiv:2406.20098, 2024

    Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, et al. Web2code: A large-scale webpage-to-code dataset and evaluation framework for multimodal llms.arXiv preprint arXiv:2406.20098, 2024

    2024 arXiv

  54. [62]

    Layoutdiffu- sion: Improving graphic layout generation by discrete diffusion probabilistic models

    Junyi Zhang, Jiaqi Guo, Shizhao Sun, Jian-Guang Lou, and Dongmei Zhang. Layoutdiffu- sion: Improving graphic layout generation by discrete diffusion probabilistic models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7226–7236, 2023. 17

    2023

  55. [63]

    Nldesign: A ui design tool for natural language interfaces

    Tianhao Zhang, Fu Peiguo, Jie Liu, Yihe Zhang, and Xingmei Chen. Nldesign: A ui design tool for natural language interfaces. InProceedings of the ACM Turing Award Celebration Conference-China 2024, pages 153–158, 2024

    2024

  56. [64]

    Bertscore: Evaluating text generation with bert.arXiv preprint arXiv:1904.09675, 2019

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert.arXiv preprint arXiv:1904.09675, 2019

    1904 arXiv

  57. [65]

    Doclayout-yolo: Enhancing document layout analysis through diverse synthetic data and global-to-local adaptive perception.arXiv preprint arXiv:2410.12628, 2024

    Zhiyuan Zhao, Hengrui Kang, Bin Wang, and Conghui He. Doclayout-yolo: Enhancing document layout analysis through diverse synthetic data and global-to-local adaptive perception.arXiv preprint arXiv:2410.12628, 2024

    2024 arXiv

  58. [66]

    Publaynet: largest dataset ever for document layout analysis

    Xu Zhong, Jianbin Tang, and Antonio Jimeno Yepes. Publaynet: largest dataset ever for document layout analysis. In2019 International conference on document analysis and recognition (ICDAR), pages 1015–1022. IEEE, 2019

    2019

  59. [67]

    Bridging design and development with automated declarative ui code generation.arXiv preprint arXiv:2409.11667, 2024

    Ting Zhou, Yanjie Zhao, Xinyi Hou, Xiaoyu Sun, Kai Chen, and Haoyu Wang. Bridging design and development with automated declarative ui code generation.arXiv preprint arXiv:2409.11667, 2024

    2024 arXiv

  60. [68]

    Judgelm: Fine-tuned large language models are scalable judges.arXiv preprint arXiv:2310.17631, 2023

    Lianghui Zhu, Xinggang Wang, and Xinlong Wang. Judgelm: Fine-tuned large language models are scalable judges.arXiv preprint arXiv:2310.17631, 2023. 18 Figure 5. The poster for this paper, powered by P2P. A. Examples of Poster Generation Examples of poster generation are shown ...

    2023 arXiv

  61. [69]

    Ground-Truth Alignment: Each checklist item references specific elements from the official academic posters and corresponding papers, ensuring accurate evaluation aligned with the original author’s intent

  62. [70]

    Domain-Specific Emphasis: The pipeline captures domain-specific expectations and conventions, which universal criteria may overlook, reflecting discipline-specific priorities

  63. [71]

    Start,"

    Essential Research Component Verification: Critical content such as key figures, method- ology details, and conclusions are explicitly accounted for using detailed scoring mecha- 19 (a) Designed by P2P (b) Designed by YuanBao RESEARCH POSTER PRESENTATION DESIGN © 2019 www.Post...

    2019

  64. [72]

    Single app, single query

  65. [73]

    The other data is created by GPT-4 instruction generated and manually constructed

    Single app, multiple queries. The other data is created by GPT-4 instruction generated and manually constructed. We obtained 103 APIs from the ADB compilation, most of which are related to tools and general buttons. As shown in the figure, our metrics are checkpoints, includin...

  66. [74]

    poster-header

    Human Preference Integration: Carefully calibrated by four human annotators, checklist item scores inherently encode domain expertise and human judgment regarding item significance and presentation quality. C. The Features of HTML Format We compare the advantages of HTML for S...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.