Pith. sign in

REVIEW 3 major objections 2 minor 75 references

The paper claims that even the strongest LLMs systematically fail on natural-language graph queries that are inherently ambiguous, as measured by a new benchmark built on an expert-verified six-cell taxonomy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A benchmark organized by a six-type taxonomy of ambiguous graph queries reportedly shows that nine LLMs, including top models, frequently produce wrong query translations.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection The submitted PDF is the wrong paper, so AmbiGraph-Eval can't be audited as-is; the abstract's benchmark idea is plausible, but the 'expert-verified answers' for inherently ambiguous queries need a real protocol before the headline claim can be trusted. the 3 major comments →

arxiv 2508.09631 v1 pith:HEYZRR7X submitted 2025-08-13 cs.DB cs.AI

AmbiGraph-Eval: Can LLMs Effectively Handle Ambiguous Graph Queries?

Yuchen Tian , Kaixin Li , Hao Chen , Ziyang Luo , Hongzhan Lin , Sebastian Schelter , Lun Du , Jing Ma This is my paper
classification cs.DB cs.AI
keywords ambiguous graph queriesLLM evaluationnatural language to query translationambiguity taxonomybenchmarkknowledge graphsgraph querying
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that large language models, including the strongest commercial systems, cannot reliably translate inherently ambiguous natural-language queries into graph queries. It introduces a taxonomy of graph-query ambiguities—attribute, relationship, and combined attribute-relationship, each appearing in same-entity and cross-entity scenarios—and builds the AmbiGraph-Eval benchmark of real-world ambiguous queries paired with expert-verified answers. Evaluating nine representative LLMs, the paper reports that even top models struggle, and that the interconnected structure of graphs amplifies the effects of ambiguity. If correct, this identifies a measurable capability gap and motivates dedicated ambiguity-resolution techniques rather than relying on general language-model competence.

Core claim

The central empirical claim is that current LLMs systematically mistranslate ambiguous natural-language graph queries, and that graph structure makes the problem worse. To demonstrate this, the paper organizes ambiguity into six cells: three types (attribute, relationship, and attribute-relationship) crossed with two scenarios (same-entity and cross-entity). Using AmbiGraph-Eval, a benchmark of real-world ambiguous queries with expert-verified graph query answers, the paper finds that both open and closed models fail across the taxonomy, with top commercial models showing only limited success. The result is positioned as a critical gap in ambiguity handling that current benchmarks and techni

What carries the argument

The organizing instrument is the six-cell ambiguity taxonomy: Attribute Ambiguity (a term could name different attribute values or labels), Relationship Ambiguity (a connection could be read as different relation types), and Attribute-Relationship Ambiguity (both at once), each under Same-Entity and Cross-Entity scenarios. This taxonomy drives benchmark construction, error categorization, and the analysis of how graph topology amplifies ambiguity.

Load-bearing premise

The load-bearing premise is that the expert-verified graph query answers used as the gold standard are genuinely correct and unambiguous readings of queries that we agree are inherently ambiguous; if experts disagree among themselves, low LLM accuracy measures disagreement with one annotator rather than a capability gap.

What would settle it

Run a second independent panel of graph-query experts to annotate the same ambiguous queries; if the two panels disagree substantially on the intended answers, or if non-expert human accuracy on the benchmark is close to LLM accuracy, the reported failure rates would reflect annotation ambiguity rather than a model-specific deficiency.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • AmbiGraph-Eval can serve as a reusable benchmark for measuring and comparing how well models handle ambiguous graph queries, not just clean translations.
  • The results indicate that accuracy on unambiguous benchmarks overstates LLM readiness for real-world graph querying, where ambiguity is frequent.
  • The taxonomy gives researchers a shared language to identify where failures concentrate, enabling targeted disambiguation methods such as clarification dialogs or schema-aware entity linking.
  • If the amplification effect is real, systems that operate on densely connected graphs need explicit ambiguity handling even when individual words appear unambiguous.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The six-cell taxonomy likely generalizes to other structured query settings, such as SPARQL or SQL over graph-like schemas, so the measured gap may apply beyond the specific graph-query task.
  • A testable extension is to inject schema or ontology context into the prompt and measure whether the performance gap narrows; the paper's taxonomy predicts that cross-entity ambiguity will respond less to such context than same-entity ambiguity.
  • Another extension is to measure human non-expert accuracy on the same queries; if humans also diverge widely, the benchmark may estimate inherent ambiguity in the queries rather than a pure model deficiency.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The submitted manuscript, as titled and abstracted, claims to introduce AmbiGraph-Eval, a benchmark of ambiguous graph queries with expert-verified answers, organized by a six-cell taxonomy (attribute, relationship, and attribute-relationship ambiguity, each in same-entity and cross-entity scenarios), and to evaluate nine representative LLMs, finding that even top models struggle. However, the full text supplied is an unrelated paper, 'Preacher: Paper-to-Video Agentic System' (arXiv:2508.09632), which describes a video-abstract generation system and contains no material about AmbiGraph-Eval, graph queries, LLM evaluation, or the claimed benchmark. As submitted, the central claims of the abstract are entirely unsupported by any accompanying methodology, data, or experimental results.

Significance. If the claimed benchmark and evaluation were present and sound, the result could be a useful contribution: a reusable taxonomy of graph-query ambiguities and an empirical demonstration of LLM difficulties could motivate specialized ambiguity-resolution techniques. The taxonomic scheme described in the abstract—attribute, relationship, and combined ambiguity, split by same-entity and cross-entity—is plausible and potentially valuable. However, no such benchmark, protocol, or evaluation is contained in the submitted full text. The paper therefore cannot be assessed for significance on its stated topic. There are no machine-checked proofs, reproducible code, or parameter-free derivations to credit; the only empirical claim is an unsupported assertion in the abstract.

major comments (3)
  1. [Full text (entire document)] The full text is not the AmbiGraph-Eval paper announced in the title and abstract. It is the Preacher paper (arXiv:2508.09632), a paper-to-video agentic system. None of the claimed components of AmbiGraph-Eval—the six-cell taxonomy, the real-world query collection, the expert-verification protocol, the nine evaluated LLMs, or the experimental results—appear anywhere in the supplied text. This is a load-bearing omission: the central claim that 'even top models struggle with ambiguous graph queries' is unverifiable and, as submitted, backed only by an abstract.
  2. [Abstract] The abstract states that the benchmark contains 'expert-verified graph query answers.' For queries defined as inherently ambiguous, the correctness of a single gold answer is not obvious: if multiple graph translations are legitimate, LLM accuracy measured against one preferred reading may reflect annotation subjectivity rather than model failure. The supplied text provides no information about the number of experts, inter-annotator agreement, scoring rubric, or whether multiple acceptable answers were allowed. This concern is untestable because the full text is absent, but it is a substantive methodological issue that any correct version of the paper must address.
  3. [Abstract] The empirical claim is stated without any quantitative support: no accuracy numbers, no model-by-model breakdown, no error bars, no baseline comparison, and no description of the evaluation protocol (prompting, temperature, decoding, scoring). The abstract's assertion that '9 representative LLMs' were evaluated and 'even top models struggle' is not accompanied by results in the submitted text. Since the full text is the wrong paper, there is no table, figure, or appendix to anchor or verify this claim.
minor comments (2)
  1. [Abstract] The abstract refers to 'real-world ambiguous queries' without specifying their source, domain, or number. If the correct manuscript is resubmitted, this should be stated explicitly.
  2. [Title and full text] The submitted file combines a title/abstract for AmbiGraph-Eval with the body of an unrelated paper. This appears to be a submission error, but it must be corrected before the work can be reviewed.

Circularity Check

0 steps flagged

No circularity detected in the available abstract; the supplied full text is a different paper, so no derivation chain could be inspected.

full rationale

The available text for arXiv:2508.09631 consists only of the abstract, which describes an empirical benchmark construction and evaluation: a taxonomy of graph-query ambiguities, a set of real-world ambiguous queries paired with expert-verified answers, and an evaluation of nine LLMs against those answers. There is no equation, fitted parameter, self-citation, or imported uniqueness theorem that would make a claimed result equivalent to an input by construction. The abstract's central claim — that even top models struggle with ambiguous graph queries — is an empirical measurement against an external gold standard, not a definitional or tautological consequence of the taxonomy. The concern raised by the skeptic about 'expert-verified' answers for queries that are inherently ambiguous is a substantive question about gold-standard validity and annotator agreement, but it is not a circularity defect under the specified patterns: the answers are not defined as 'whatever the LLM outputs' nor derived from the models under test. Moreover, the supplied full text is actually the unrelated Preacher paper (arXiv:2508.09632), so no additional internal reduction or circular step can be examined. Under the hard rule that circularity must be exhibited by specific quoted reductions, the honest finding is no significant circularity, score 0.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 1 invented entities

This ledger is abstract-only. The central result depends entirely on benchmark construction choices the abstract does not report: how many and which real-world queries were selected, how 'inherently ambiguous' was determined, how many experts verified answers and at what agreement threshold, and how the nine LLMs were prompted and scored. The taxonomy is asserted as the organizing scheme without a completeness argument. None of these are demonstrated errors; they are unverifiable premises that any reader of the abstract would need the full paper to audit.

free parameters (3)
  • Benchmark query set composition (number, domains, sources of real-world queries) = not reported in abstract
    The 'top models struggle' result depends entirely on which real-world ambiguous queries were selected and how many fall in each taxonomy cell; the abstract reports no counts or sources.
  • Expert-verification protocol (number of experts, agreement threshold) = not reported in abstract
    The central claim depends on gold answers being expert-verified, but the abstract states neither how many experts were involved nor what agreement level was required.
  • LLM evaluation protocol (prompting scheme, temperature, decoding, scoring thresholds) = not reported in abstract
    Comparing nine LLMs fairly requires a defined evaluation protocol; the abstract does not state how queries were posed or how 'struggling' was quantified.
axioms (3)
  • domain assumption The six-cell taxonomy (attribute/relationship/attribute-relationship across same-entity and cross-entity scenarios) is an exhaustive and natural partition of ambiguity in graph queries.
    Benchmark validity rests on the taxonomy capturing genuine ambiguity classes; if real ambiguity does not fall into these cells, the benchmark measures constructed edge cases. Stated in the abstract's taxonomy description.
  • domain assumption The collected queries are genuinely 'inherently ambiguous' in real usage, not merely imprecise or under-specified.
    The abstract claims 'real-world queries often contain inherent ambiguities'; the benchmark needs these queries to be representative of real workloads rather than examples constructed to fit the taxonomy.
  • domain assumption Standard LLM evaluation methodology (sampling, prompting, scoring) is applied uniformly and fairly across the nine models.
    The comparative finding requires apples-to-apples evaluation, which the abstract does not describe.
invented entities (1)
  • Six-cell ambiguity taxonomy (attribute, relationship, attribute-relationship; each with same-entity and cross-entity scenarios) no independent evidence
    purpose: Defines the measurement categories around which the benchmark is constructed.
    The taxonomy is a new conceptual apparatus introduced by the paper. The abstract provides no external falsifiable handle, such as data showing that real query workloads distribute across these cells; its completeness is asserted rather than demonstrated.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of AmbiGraph-Eval: Can LLMs Effectively Handle Ambiguous Graph Queries?." pith.science (2026). https://pith.science/paper/HEYZRR7X

@misc{pith2026250809631,
  author       = {Pith},
  title        = {Pith review of: AmbiGraph-Eval: Can LLMs Effectively Handle Ambiguous Graph Queries?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HEYZRR7X}},
  note         = {Machine review of arXiv:2508.09631}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Models (LLMs) have recently demonstrated strong capabilities in translating natural language into database queries, especially when dealing with complex graph-structured data. However, real-world queries often contain inherent ambiguities, and the interconnected nature of graph structures can amplify these challenges, leading to unintended or incorrect query results. To systematically evaluate LLMs on this front, we propose a taxonomy of graph-query ambiguities, comprising three primary types: Attribute Ambiguity, Relationship Ambiguity, and Attribute-Relationship Ambiguity, each subdivided into Same-Entity and Cross-Entity scenarios. We introduce AmbiGraph-Eval, a novel benchmark of real-world ambiguous queries paired with expert-verified graph query answers. Evaluating 9 representative LLMs shows that even top models struggle with ambiguous graph queries. Our findings reveal a critical gap in ambiguity handling and motivate future work on specialized resolution techniques.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

75 extracted references · 53 canonical work pages · 1 internal anchor

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    Pith/arXiv arXiv

  2. [2]

    A multi-agent deep reinforcement learning ap- proach for enhancement of covid-19 ct image segmentation

    Hanane Allioui, Mazin Abed Mohammed, Narjes Be- nameur, Belal Al-Khateeb, Karrar Hameed Abdulkareem, Begonya Garcia-Zapirain, Robertas Damaˇseviˇcius, and Rytis Maskeli¯unas. A multi-agent deep reinforcement learning ap- proach for enhancement of covid-19 ct image segmentation. Journal of personalized medicine, 12(2):309, 2022. 3

    work page 2022

  3. [3]

    Prompt augmentation for self-supervised text-guided image manipulation

    Rumeysa Bodur, Binod Bhattarai, and Tae-Kyun Kim. Prompt augmentation for self-supervised text-guided image manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8829– 8838, 2024. 3

    work page 2024

  4. [4]

    Video abstracts are associ- ated with an increase in research reports citations, views and social attention: a cross-sectional study

    Tristan Bonnevie, Aurore Repel, Francis-Edouard Gravier, Joel Ladner, Louis Sibert, Jean-Franc ¸ois Muir, Antoine Cu- velier, and Marc-Olivier Fischer. Video abstracts are associ- ated with an increase in research reports citations, views and social attention: a cross-sectional study. Scientometrics, 128 (5):3001–3015, 2023. 2

    work page 2023

  5. [5]

    A reinforced lunar dynamo recorded by chang’e-6 farside basalt

    Shuhui Cai, Kaixian Qi, Saihong Yang, Jie Fang, Pingyuan Shi, Zhongshan Shen, Min Zhang, Huafeng Qin, Chi Zhang, Xiaoguang Li, et al. A reinforced lunar dynamo recorded by chang’e-6 farside basalt. Nature, pages 1–3, 2024. 6

    work page 2024

  6. [6]

    Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7310– 7320, 2024. 3

    work page 2024

  7. [7]

    Hallo2: Long-duration and high-resolution audio-driven portrait im- age animation

    Jiahao Cui, Hui Li, Yao Yao, Hao Zhu, Hanlin Shang, Kaihui Cheng, Hang Zhou, Siyu Zhu, and Jingdong Wang. Hallo2: Long-duration and high-resolution audio-driven portrait im- age animation. arXiv preprint arXiv:2410.07718, 2024. 4

    Pith/arXiv arXiv 2024

  8. [8]

    Collaborating with language models for embodied reason- ing

    Ishita Dasgupta, Christine Kaeser-Chen, Kenneth Marino, Arun Ahuja, Sheila Babayan, Felix Hill, and Rob Fergus. Collaborating with language models for embodied reason- ing. In NeurIPS 2022 Foundation Models for Decision Mak- ing Workshop. 3

    work page 2022

  9. [9]

    Deepseek-r1: Incentivizing reasoning capa- bility in llms via reinforcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capa- bility in llms via reinforcement learning, 2025. 7

    work page 2025

  10. [10]

    Lon- grope: extending llm context window beyond 2 million to- kens

    Yiran Ding, Li Lyna Zhang, Chengruidong Zhang, Yuanyuan Xu, Ning Shang, Jiahang Xu, Fan Yang, and Mao Yang. Lon- grope: extending llm context window beyond 2 million to- kens. JMLR.org, 2025. 2

    work page 2025

  11. [11]

    Self- collaboration code generation via chatgpt

    Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. Self- collaboration code generation via chatgpt. ACM Transac- tions on Software Engineering and Methodology , 33(7):1– 38, 2024. 3

    work page 2024

  12. [12]

    Cosyvoice 2: Scalable streaming speech synthe- sis with large language models

    Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, et al. Cosyvoice 2: Scalable streaming speech synthe- sis with large language models. CoRR, 2024. 7

    work page 2024

  13. [13]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 ,

    Pith/arXiv arXiv

  14. [14]

    Audio-visual tools in science communication: the video abstract in ecology and environ- mental sciences

    Miguel Ferreira, Betina Lopes, Ant ´onio Granado, Helena Freitas, and Jo ˜ao Loureiro. Audio-visual tools in science communication: the video abstract in ecology and environ- mental sciences. Frontiers in Communication , 6:596248,

    work page

  15. [15]

    Comfygen: Prompt- adaptive workflows for text-to-image generation

    Rinon Gal, Adi Haviv, Yuval Alaluf, Amit H Bermano, Daniel Cohen-Or, and Gal Chechik. Comfygen: Prompt- adaptive workflows for text-to-image generation. arXiv preprint arXiv:2410.01731, 2024. 3

    Pith/arXiv arXiv 2024

  16. [16]

    High-fidelity and freely controllable talking head video generation

    Yue Gao, Yuan Zhou, Jinglu Wang, Xiao Li, Xiang Ming, and Yan Lu. High-fidelity and freely controllable talking head video generation. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 5609–5619, 2023. 4

    work page 2023

  17. [17]

    Large language model based multi- agents: A survey of progress and challenges

    T Guo, X Chen, Y Wang, R Chang, S Pei, NV Chawla, O Wiest, and X Zhang. Large language model based multi- agents: A survey of progress and challenges. In 33rd Inter- national Joint Conference on Artificial Intelligence (IJCAI 2024). IJCAI; Cornell arxiv, 2024. 3

    work page 2024

  18. [18]

    An integrated guide for designing video abstracts using freeware and their emerging role in academic research advancement

    Ria Gupta, Mrudula Joshi, and Latika Gupta. An integrated guide for designing video abstracts using freeware and their emerging role in academic research advancement. Journal of Korean Medical Science, 36(9), 2021. 1, 3

    work page 2021

  19. [19]

    Streamingt2v: Con- sistent, dynamic, and extendable long video generation from text

    Roberto Henschel, Levon Khachatryan, Daniil Hayrapetyan, Hayk Poghosyan, Vahram Tadevosyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Streamingt2v: Con- sistent, dynamic, and extendable long video generation from text. arXiv preprint arXiv:2403.14773, 2024. 3, 6, 7

    Pith/arXiv arXiv 2024

  20. [20]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022. 2

    2022

  21. [21]

    Metagpt: Meta pro- gramming for a multi-agent collaborative framework

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. Metagpt: Meta pro- gramming for a multi-agent collaborative framework. InThe Twelfth International Conference on Learning Representa- tions. 3, 5

    work page

  22. [22]

    Inner monologue: Em- bodied reasoning through planning with language models

    Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Em- bodied reasoning through planning with language models. In Conference on Robot Learning, pages 1769–1782. PMLR,

    work page

  23. [23]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richard- son, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024. 6

    Pith/arXiv arXiv 2024

  24. [24]

    A comprehensive survey on process-oriented automatic text summarization with exploration of llm-based methods

    Hanlei Jin, Yang Zhang, Dan Meng, Jun Wang, and Jinghua Tan. A comprehensive survey on process-oriented automatic text summarization with exploration of llm-based methods. arXiv preprint arXiv:2403.02901, 2024. 2

    arXiv 2024

  25. [25]

    Agentreview: Explor- ing peer review dynamics with llm agents

    Yiqiao Jin, Qinlin Zhao, Yiyang Wang, Hao Chen, Kaijie Zhu, Yijia Xiao, and Jindong Wang. Agentreview: Explor- ing peer review dynamics with llm agents. InProceedings of the 2024 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1208–1226, 2024. 2

    work page 2024

  26. [26]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15954–15964, 2023. 3

    2023

  27. [27]

    �����������������������������

    Kling. ����������������������������� . 7

    work page

  28. [28]

    A human-inspired reading agent with gist memory of very long contexts

    Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta, John Canny, and Ian Fischer. A human-inspired reading agent with gist memory of very long contexts. In Proceedings of the 41st International Conference on Machine Learning . JMLR.org,

    work page

  29. [29]

    Literature reviews with llm-based tools

    Hugo Letiche and Michael Lissack. Literature reviews with llm-based tools. Available at SSRN 5110658. 2

    work page

  30. [30]

    Camel: Communicative agents for” mind” exploration of large language model society

    Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for” mind” exploration of large language model society. Advances in Neural Information Processing Systems , 36: 51991–52008, 2023. 3

    work page 2023

  31. [31]

    Jiaqi Li, Mengmeng Wang, Zilong Zheng, and Muhan Zhang. Loogle: Can long-context language models under- stand long contexts? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16304–16333, 2024. 2

    work page 2024

  32. [32]

    Long-context llms struggle with long in-context learn- ing

    Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue, and Wenhu Chen. Long-context llms struggle with long in-context learn- ing. CoRR, 2024. 2, 4

    work page 2024

  33. [33]

    ���������������������������������

    Luma. ��������������������������������� . 3, 4, 7

    work page

  34. [34]

    �������������������������������

    Midjourney. ������������������������������� . 2

    work page

  35. [35]

    Sora: Creating video from text

    OpenAI. Sora: Creating video from text. �������� ��������������� , 2024. 2, 3, 4, 5, 6, 7

    work page 2024

  36. [36]

    ������������������������� ���������������

    OpenAIO3-mini. ������������������������� ��������������� . 6, 7

    work page

  37. [37]

    ���������������������

    PIKA. ��������������������� . 2, 3, 4

    work page

  38. [38]

    Commu- nicative agents for software development

    Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Commu- nicative agents for software development. CoRR, 2023. 3

    work page 2023

  39. [39]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, pages 8748–8763. PmLR, 2021. 3, 7

    work page 2021

  40. [40]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. 4

    work page

  41. [41]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 4

    2022

  42. [42]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234–241. Springer, 2015. 3

    2015

  43. [43]

    The emergence of large language models (llm) as a tool in literature re- views: an llm automated systematic review

    Dmitry Scherbakov, Nina Hubig, Vinita Jansari, Alexan- der Bakumenko, and Leslie A Lenert. The emergence of large language models (llm) as a tool in literature re- views: an llm automated systematic review. arXiv preprint arXiv:2409.04600, 2024. 2

    Pith/arXiv arXiv 2024

  44. [44]

    Laion- 400m: Open dataset of clip-filtered 400 million image-text pairs

    Christoph Schuhmann, Robert Kaczmarczyk, Aran Komat- suzaki, Aarush Katta, Richard Vencu, Romain Beaumont, Jenia Jitsev, Theo Coombes, and Clayton Mullis. Laion- 400m: Open dataset of clip-filtered 400 million image-text pairs. In NeurIPS Workshop Datacentric AI , number FZJ- 2022-00923. J¨ulich Supercomputing Center, 2021. 2

    work page 2022

  45. [45]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural in- formation processing systems, 35:25278–25294, 2022. 2, 7

    work page 2022

  46. [46]

    Llm-planner: Few-shot grounded planning for embodied agents with large language models

    Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. Llm-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 2998–3009, 2023. 3

    work page 2023

  47. [47]

    �����������������������������

    tavus. ����������������������������� . 4, 7

    work page

  48. [48]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. 4, 7

    Pith/arXiv arXiv 2023

  49. [49]

    Overview of the nlpcc2024 shared task 6: Scientific literature survey generation

    Yangjie Tian, Xungang Gu, Aijia Li, He Zhang, Ruohua Xu, Yunfeng Li, and Ming Liu. Overview of the nlpcc2024 shared task 6: Scientific literature survey generation. InCCF International Conference on Natural Language Processing and Chinese Computing, pages 400–408. Springer, 2024. 2

    work page 2024

  50. [50]

    Videotetris: Towards compositional text-to- video generation

    Ye Tian, Ling Yang, Haotian Yang, Yuan Gao, Yufan Deng, Xintao Wang, Zhaochen Yu, Xin Tao, Pengfei Wan, Di ZHANG, et al. Videotetris: Towards compositional text-to- video generation. Advances in Neural Information Process- ing Systems, 37:29489–29513, 2024. 2, 3, 6

    work page 2024

  51. [51]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 3, 4

    Pith/arXiv arXiv 2023

  52. [52]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023. 4

    Pith/arXiv arXiv 2023

  53. [53]

    SPAgent: Adaptive Task Decomposition and Model Selection for General Video Generation and Editing

    Rong-Cheng Tu, Wenhao Sun, Zhao Jin, Jingyi Liao, Jiaxing Huang, and Dacheng Tao. Spagent: Adaptive task decompo- sition and model selection for general video generation and editing. arXiv preprint arXiv:2411.18983, 2024. 3

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  54. [54]

    Uhlenbeck and S

    K. Uhlenbeck and S. T. Yau. On the existence of hermitian- yang-mills connections in stable vector bundles. Communi- cations on Pure and Applied Mathematics, 1985. 6

    work page 1985

  55. [55]

    Rectified diffusion: Straightness is not your need in rectified flow

    Fu-Yun Wang, Ling Yang, Zhaoyang Huang, Mengdi Wang, and Hongsheng Li. Rectified diffusion: Straightness is not your need in rectified flow. In International Conference on Learning Representations, 2025. 2

    work page 2025

  56. [56]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024. 2

    Pith/arXiv arXiv 2024

  57. [57]

    Videocomposer: Compositional video synthesis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. Videocomposer: Compositional video synthesis with motion controllability. Advances in Neural Information Processing Systems, 36:7594–7611, 2023. 3

    2023

  58. [58]

    Autosurvey: Large language models can automatically write surveys

    Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Qingsong Wen, Wei Ye, et al. Autosurvey: Large language models can automatically write surveys. In The Thirty-eighth An- nual Conference on Neural Information Processing Systems. 2

    work page

  59. [59]

    Genartist: Multimodal llm as an agent for unified image gen- eration and editing

    Zhenyu Wang, Aoxue Li, Zhenguo Li, and Xihui Liu. Genartist: Multimodal llm as an agent for unified image gen- eration and editing. Advances in Neural Information Pro- cessing Systems, 37:128374–128395, 2025. 3

    work page 2025

  60. [60]

    ����� � � � ������ � ��� � ��� � ������������

    Tongyi Wanxiang. ����� � � � ������ � ��� � ��� � ������������ . 4, 6, 7

    work page

  61. [61]

    Art-v: Auto-regressive text-to- video generation with diffusion models

    Wenming Weng, Ruoyu Feng, Yanhui Wang, Qi Dai, Chunyu Wang, Dacheng Yin, Zhiyuan Zhao, Kai Qiu, Jian- min Bao, Yuhui Yuan, et al. Art-v: Auto-regressive text-to- video generation with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7395–7405, 2024. 3

    work page 2024

  62. [62]

    A survey on llm- generated text detection: Necessity, methods, and future di- rections

    Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. A survey on llm- generated text detection: Necessity, methods, and future di- rections. Computational Linguistics, pages 1–66, 2025. 2

    work page 2025

  63. [63]

    Dreamfactory: Pio- neering multi-scene long video generation with a multi-agent framework

    Zhifei Xie, Daniel Tang, Dingwei Tan, Jacques Klein, Tegawend F Bissyand, and Saad Ezzini. Dreamfactory: Pio- neering multi-scene long video generation with a multi-agent framework. arXiv preprint arXiv:2408.11788, 2024. 3

    Pith/arXiv arXiv 2024

  64. [64]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open-domain images with video diffusion priors. In Eu- ropean Conference on Computer Vision , pages 399–417. Springer, 2025. 3

    work page 2025

  65. [65]

    Improving diffusion-based image synthesis with context pre- diction

    Ling Yang, Jingwei Liu, Shenda Hong, Zhilong Zhang, Zhilin Huang, Zheming Cai, Wentao Zhang, and Bin Cui. Improving diffusion-based image synthesis with context pre- diction. Advances in Neural Information Processing Sys- tems, 36:37636–37656, 2023. 2

    work page 2023

  66. [66]

    Mastering text-to-image diffu- sion: Recaptioning, planning, and generating with multi- modal llms

    Ling Yang, Zhaochen Yu, Chenlin Meng, Minkai Xu, Ste- fano Ermon, and Bin Cui. Mastering text-to-image diffu- sion: Recaptioning, planning, and generating with multi- modal llms. In Forty-first International Conference on Ma- chine Learning, 2024. 2

    work page 2024

  67. [67]

    Mmada: Mul- timodal large diffusion language models

    Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Mul- timodal large diffusion language models. arXiv preprint arXiv:2505.15809, 2025. 2

    Pith/arXiv arXiv 2025

  68. [68]

    Mora: Enabling gen- eralist video generation via a multi-agent framework

    Zhengqing Yuan, Ruoxi Chen, Zhaoxu Li, Haolong Jia, Li- fang He, Chi Wang, and Lichao Sun. Mora: Enabling gen- eralist video generation via a multi-agent framework. arXiv preprint arXiv:2403.13248, 2024. 3

    Pith/arXiv arXiv 2024

  69. [69]

    Make pixels dance: High- dynamic video generation

    Yan Zeng, Guoqiang Wei, Jiani Zheng, Jiaxin Zou, Yang Wei, Yuchen Zhang, and Hang Li. Make pixels dance: High- dynamic video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8850–8860, 2024. 3

    work page 2024

  70. [70]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145, 2023. 3

    Pith/arXiv arXiv 2023

  71. [71]

    Ppta- gent: Generating and evaluating presentations beyond text- to-slides

    Hao Zheng, Xinyan Guan, Hao Kong, Jia Zheng, Hongyu Lin, Yaojie Lu, Ben He, Xianpei Han, and Le Sun. Ppta- gent: Generating and evaluating presentations beyond text- to-slides. arXiv preprint arXiv:2501.03936, 2025. 3

    Pith/arXiv arXiv 2025

  72. [72]

    Chatgpt research group for optimizing the crystallinity of mofs and cofs

    Zhiling Zheng, Oufan Zhang, Ha L Nguyen, Nakul Rampal, Ali H Alawadhi, Zichao Rong, Teresa Head-Gordon, Chris- tian Borgs, Jennifer T Chayes, and Omar M Yaghi. Chatgpt research group for optimizing the crystallinity of mofs and cofs. ACS Central Science, 9(11):2161–2170, 2023. 6, 7

    work page 2023

  73. [73]

    Is llm a reliable re- viewer? a comprehensive evaluation of llm on automatic pa- per reviewing tasks

    Ruiyang Zhou, Lu Chen, and Kai Yu. Is llm a reliable re- viewer? a comprehensive evaluation of llm on automatic pa- per reviewing tasks. In Proceedings of the 2024 Joint In- ternational Conference on Computational Linguistics, Lan- guage Resources and Evaluation (LREC-COLING 2024) , pages 9340–9351, 2024. 2

    work page 2024

  74. [74]

    An intelligent agentic system for complex image restoration problems

    Kaiwen Zhu, Jinjin Gu, Zhiyuan You, Yu Qiao, and Chao Dong. An intelligent agentic system for complex image restoration problems. CoRR, 2024. 3

    work page 2024

  75. [75]

    The impact of video abstract on citation counts: evidence from a retrospective cohort study of new journal of physics

    Qianjin Zong, Yafen Xie, Rongchan Tuo, Jingshi Huang, and Yang Yang. The impact of video abstract on citation counts: evidence from a retrospective cohort study of new journal of physics. Scientometrics, 119:1715–1727, 2019. 2

    work page 2019

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.