AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design

Haobin Jiang; Haodong Li; Jiacheng Cui; Jiacheng Liu; Jialv Zou; Jing Li; Wenhao Yan; Xiaofu Chen; Xiaohan Zhao; Xiaotong Li

REVIEW 3 major objections 5 minor 18 references

AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AutoDesign claims the harness around a fixed model—not the model itself—is the right optimization target, and demonstrates it with a 78.32 score on the 100-paper PosterBench Main Track, 7.45 points above Claude Design.

desk verdict A genuinely reusable harness-optimization recipe and a new benchmark, but the headline results are not yet interpretable because the paper never states that the train/dev sets are disjoint from the evaluation sets. read the letter →

arxiv 2608.13560 v1 pith:DWZGCBZY submitted 2026-08-13 cs.CV cs.AIcs.CL

Yaxin Luo , Haobin Jiang , Jialv Zou , Xu Huang , Wenhao Yan , Haodong Li , Zhengrong Yue , Jing Li

show 6 more authors

Xiaofu Chen Xiaohan Zhao Jiacheng Liu Jiacheng Cui Zhiqiang Shen Xiaotong Li

This is my paper · ORCID

classification cs.CVcs.AIcs.CL

keywords meta-harnessoptimizationdesignharnesspaper-to-postergenerationPosterBenchlong-horizonagenticrecursiveself-improvementmultimodalacceptancegate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AutoDesign is trying to establish that a multimodal design system can improve itself by optimizing the 'harness'—the prompts, tools, validation rules, and control flow surrounding a fixed model—rather than the model's weights. The paper proposes two nested loops: an inner loop in which the harness converts a source paper into an editable poster through repeated critique-and-repair, and an outer loop in which a meta-harness studies recorded rollouts and scores, proposes a change to exactly one of five harness components, and accepts the change only if training performance rises and development performance does not fall. To test the idea, the paper builds PosterBench, a 100-paper benchmark with a seven-dimension rubric, and reports that the evolved DesignHarness raises the average PosterBench Score across seven model/coding-agent configurations from 54.99 to 67.39, reaches 78.32 on the Main Track, and wins a system-blind human preference study. A sympathetic reader would care because the result suggests that durable, transferable design skill can be accumulated in software around a model at low cost, without retraining the model.

What carries the argument

The load-bearing mechanism is the design harness $H$ with its five-component decomposition, together with the two-loop meta-harness cycle built on it. The inner loop instantiates the harness as a designer–critic pair, $y_k = M^{\mathrm{design}}(y_{k-1}, f_{k-1}; x, c)$ and $f_k = M^{\mathrm{critic}}(y_k; x, c)$, so the artifact is refined locally as editable code while rule-based validation and vision-language-model critique supply repair feedback. The outer loop aggregates trajectories $\tau_t$ and scores $s_t$ across training tasks, and a coding agent acting as the meta-harness optimizer $P$ proposes one component-restricted update from this evidence plus an optimization record $L$; the acceptance gate then promotes the candidate only when $J_{\text{train}}(H') > J_{\text{train}}(H)$ and $J_{\text{dev}}(H') \geq J_{\text{dev}}(H)$, with the development set kept hidden from the optimizer so that it guards against overfitting the harness to training papers. What carries the empirical argument is PosterBench, a frozen, externally specified evaluator producing a weighted seven-dimension rubric score (faithfulness, coverage, density, visual evidence, layout, readability, aesthetics) with record-level ceilings, applied after optimization so that the optimized harness and its competitors are measured under the same protocol.

What would settle it

Check the released per-case records: if any of the 100 Main Track case IDs (for example '2017-attention-is-all-you-need') also appears in the training set $D_{\text{train}}$ or development set $D_{\text{dev}}$ of Algorithm 1, the reported 78.32 is not an unbiased held-out score. A complementary check is to rerun the meta-harness loop with the acceptance gate disabled; if the ungated harness scores about the same on papers outside the optimization sets, the development-set criterion is not what prevents overfitting.

Watch

Extended reading notes

Core claim

The paper's central claim is that a static production procedure for multimodal output can be turned into a persistent learning system by making the design harness itself the object of optimization. Formally, the harness is $H$ with $y \sim H(\pi_\theta, x, c)$ where the model parameters $\theta$ stay fixed and the optimization acts on the surrounding system, decomposed into five functional components: context and memory, tools and specifications, execution runtime, orchestration, and evaluation and feedback. The meta-harness objective is $H^\star = \arg\max_H J(H)$ with $J(H)$ the expected quality of produced artifacts under a human-aligned evaluator; each outer-loop iteration restricts the update to one component and gates promotion on $J_{\text{train}}(H') > J_{\text{train}}(H)$ and $J_{\text{dev}}(H') \geq J_{\text{dev}}(H)$. Iterating this loop produces DesignHarness, an executable paper-to-poster system whose designer module authors editable HTML, validates it with rule-based blocking checks, critiques rendered previews with a vision-language model, and finalizes the best valid candidate. On the newly introduced PosterBench protocol, the claim is confirmed across seven configurations: attaching DesignHarness improves every one of them by 5.0 to 19.6 points, and the full system scores 78.32, surpassing the commercial Claude Design by 7.45 points and receiving the highest Bradley–Terry preference (64.0%) in a system-blind human study.

Load-bearing premise

The load-bearing assumption is that the 100 papers scored in the PosterBench Main Track are disjoint from the training and development papers used to optimize and gate the harness, so the headline 78.32 is a held-out measurement; the paper states the training/development split in Algorithm 1 but never explicitly says the Main Track excludes those papers.

Editorial extensions

If this is right

Harness capability transfers across models and coding agents: the same DesignHarness improves all seven tested configurations, so the learned system-level skill is not tied to a single model.
High-quality posters become cheap: with the LongCat 2.0 configuration the harness reaches 55.13 at roughly $0.27 per poster, making conference-poster quality accessible without frontier-model prices.
PosterBench scores are most informative when the gap is large: blind human agreement with the benchmark-preferred poster rises from 51.9% for 0–3 point gaps to 74.4% for gaps of at least 20 points.
The same meta-harness loop is claimed to extend to slides, webpages, and videos, provided each medium gets its own evaluator, rendering checks, and promotion gate; these outputs are pilots in this paper.
Because posters remain editable HTML with provenance links to the source paper, a user can apply localized revisions after generation instead of regenerating the whole artifact.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The gain pattern in the seven configurations—largest on the weakest models (+19.6 for DeepSeek V4 Pro, +17.8 for Seed 2.1 Pro, versus +5.0 and +5.6 for Claude 4.8 and GPT-5.5)—suggests harness optimization acts partly as a skill-transfer mechanism, a pattern worth probing on other design tasks.
A direct ablation would settle the acceptance gate's role: rerun the outer loop with the development-set criterion disabled and compare final scores; if the ungated harness matches the gated one, the gate is not what prevents overfitting.
The paper treats the optimization record $L$ as the system's memory; wiping $L$ and rerunning optimization from the initial harness would quantify how much of the final gain is accumulated experience rather than the initial scaffold.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, this is the most complete attempt I've seen at turning the harness-optimization idea into a concrete artifact for a real design task, and it ships code, data records, and a benchmark. Second, the headline number is not yet trustworthy, because the paper never states that the 100-paper Main Track and the 10-paper mini subset are disjoint from the training and development sets used in Algorithm 1.

What is actually new: the five-component decomposition of a design harness is a useful organizational device; the bounded one-component update with a train/dev acceptance gate is a clean recipe; and PosterBench's seven-dimension protocol with protected gates is a serious attempt at a hard-to-evaluate task. Seven configurations all improve with the learned harness. Combined with the released repository and per-case record schema, I believe the qualitative claim that the harness helps. That deserves real credit.

The soft spots, in proportion. The big one is data provenance. Section 3.2 says Ddev is "independent," but only relative to Dtrain. Algorithm 1 never mentions PosterBench. Appendix A.1's comparison matrix lists fixed factors, but not whether the optimization sets were drawn from the same papers. So the 78.32 Main Track could be an optimization-set score, and the +5 to +19.6 gains on mini could be in-sample. That is the difference between a generalization result and tuning to the test set. It is not a minor wording issue. Also, the same team designed Rmeta and PosterBench with the same seven dimension names. The paper says the functions are separate, but the shared vocabulary and authorship make the split question more pressing, not less. Second, the headline 78.32 has no uncertainty estimate; the human study's bootstrap interval is wide and the nominal Krippendorff coefficient is 0.101. The human evidence is weak, though the paper is honest about it.

None of this kills the core idea. The recipe is plausible, the artifacts are concrete, and the benchmark could be widely used. But the authors need to state explicitly which papers are in Dtrain and Ddev, whether either set overlaps PosterBench or PosterBench-mini, and release the assignment. If they do, this is a useful contribution; if they cannot, the numbers are not interpretable.

Who is this for: people building self-improving agent harnesses and anyone evaluating multimodal generation systems. PosterBench alone may be worth the paper. I would send it to peer review, and a serious referee should demand the data-split clarification before acceptance.

Referee Report

3 major / 5 minor

Summary. The manuscript introduces AutoDesign, a two-loop meta-harness optimization framework that iteratively improves a design harness (prompts, tools, orchestration, validation, and evaluation feedback) around a fixed model, and applies it to the academic paper-to-poster task. It also introduces PosterBench, a 100-paper Main Track plus a 10-paper PosterBench-mini subset, with a seven-dimension rubric spanning faithfulness, coverage, density, visual evidence, layout, readability, and aesthetics. The paper reports that AutoDesign with DesignHarness scores 78.32 on the Main Track, 7.45 points above the commercial Claude Design system; that attaching DesignHarness improves all seven tested model/coding-agent configurations by between 5.0 and 19.6 points, raising the average PosterBench Score from 54.99 to 67.39; and that a system-blind human study gives AutoDesign the highest Bradley-Terry preference estimate at 64.0%. The central evaluation claim is that these numbers reflect generalization of the learned harness, which requires the optimization sets Dtrain and Ddev from Algorithm 1 to be disjoint from the PosterBench evaluation sets. The manuscript does not state that this disjointness holds.

Significance. If the reported results are genuinely held-out, the paper makes a useful contribution: it demonstrates a concrete instantiation of harness-level optimization with persistent updates, provides a multi-dimension benchmark with a documented protocol, and ships released evaluation records that support auditing. The controlled ablation across seven configurations showing consistent gains, and the cost-performance analysis, are valuable even apart from the headline number. The main uncertainties are data separation and evaluator alignment: the absence of an explicit disjointness statement for Dtrain/Ddev versus PosterBench, and the fact that the optimization-time evaluator Rmeta and the final PosterBench metric share the same seven-dimension vocabulary and were both authored by the same team. Until those points are resolved, the headline 78.32 and the +12.4-point average gain are not established as unbiased generalization results.

major comments (3)

[3.2 / Algorithm 1 / 5.1 / A.1] Algorithm 1 optimizes the harness against Dtrain and gates updates with Ddev, but the manuscript never states that either set is disjoint from the 100-paper PosterBench Main Track or the 10-paper PosterBench-mini. The 'independent development set' in Section 3.2 is described only as guarding against overfitting to Dtrain, not as being held out from PosterBench; Section A.1 and Table 5 list the source paper as a fixed factor but do not exclude optimization-set papers. Consequently, the headline Main Track score 78.32 (Section 5.1) and the average gain from 54.99 to 67.39 (Section 5.2.1, Table 4) are unbiased only under an unstated assumption. The authors must state the split explicitly; if any overlap exists, the evaluations must be re-run on disjoint papers or the affected numbers must be reported as in-sample results.
[3.2 / A.4 / Table 6] The paper separates Rmeta from PosterBench by role ('Rmeta supplies feedback... PosterBench evaluates completed systems'), but both evaluators are authored by the same team and use the same seven-dimension vocabulary with the same weight emphasis on density, layout, and readability. Optimizing against Rmeta therefore implicitly optimizes toward PosterBench, so consistent gains across seven configurations do not by themselves establish that the harness improves human-judged quality. Please provide a quantitative check of the coupling, such as the correlation between Rmeta and PosterBench scores on a held-out sample of artifacts, or an ablation using an independently specified evaluator, so that readers can assess the degree of metric alignment.
[5.3 / A.5 / 1] The abstract and Section 1 claim that AutoDesign 'reaches average conference-poster quality in human evaluation,' but the human study in Section 5.3 is a system-blind pairwise preference comparison among four systems, not a rating of posters against real conference posters. The reported benchmark-human agreement (r = 0.34) and the nominal Krippendorff alpha of 0.101 in Appendix A.5 are modest. The pairwise-preference claim is supported by the data, but the 'conference-poster quality' claim should be removed or replaced with a direct human rating against real conference posters.

minor comments (5)

[5.1] The 78.32 Main Track score is reported without any uncertainty estimate; given N=100 and the visible per-dimension variance, a bootstrap confidence interval would clarify the comparison with the 70.87 Claude Design baseline.
[Abstract / 1] The gain is reported as '+12.4%' in the abstract and as '+12.40 points' in Section 1; please use consistent units.
[Figure 1(a)] The trajectory values (49.00, 80.88, 88.39) are not linked to a specific source paper or configuration; the caption or appendix should give the case id and the model/coding-agent details.
[5.3] The 64.0% Bradley-Terry estimate should be accompanied in the main text by its 95% interval (55.2–77.8%), since that interval overlaps the 51.7% point estimate for Claude Code and readers may otherwise over-interpret the gap.
[A.5] The note that Krippendorff's alpha is a nominal agreement diagnostic rather than the ranking estimator is useful but appears only in the appendix; one sentence in Section 5.3 would prevent misinterpretation.

Circularity Check

0 steps flagged · score 1.0 of 10

No demonstrated circularity: the headline PosterBench score is an empirical evaluation, not a quantity entailed by the optimization equations; the unstated disjointness of Dtrain/Ddev from PosterBench is a correctness risk, not a circular reduction.

full rationale

The paper's derivation chain is a standard meta-optimization loop: the objective J(H) is defined through the optimization-time evaluator Rmeta (Eq. 2), while the headline 78.32 is measured with the separate frozen PosterBench protocol (Eqs. 7-8). The paper explicitly states that "PosterBench is a frozen external evaluator, separate from the optimization-time evaluator Rmeta used in the meta-harness outer loop" (Section 5.1) and that the seven dimensions "use the same quality vocabulary as the outer-loop evaluator Rmeta ... but serve a different role" (Section A.4). Sharing a rubric vocabulary is a design overlap, not an identity: no equation in the paper sets Rmeta equal to Rposter, so the benchmark score is not equal to the optimized objective by construction. The main validity threat is that Algorithm 1's Dtrain and Ddev are never stated to be disjoint from PosterBench Main Track or PosterBench-mini; if they overlapped, the reported gains would be in-sample. Section 3.2 only says Ddev "serves as a guard against overfitting the harness to the training tasks," and Section 5.1 separates evaluator functions rather than data instances. This is an important omitted specification and a correctness/overfitting risk, but it is not a demonstrated circular reduction: the paper does not define Dtrain or Ddev in terms of the evaluation set, and no fitted parameter is renamed as a prediction. There are no load-bearing self-citations by the current authors, and the system-blind human study provides some independent grounding. Under the hard rules requiring an exhibited reduction, I find no significant circularity and score 1.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim depends on manually chosen rubric weights and gate caps, on the unstated independence of the optimization and evaluation paper sets, and on the validity of the self-authored evaluator. No new physical or conceptual entities are postulated beyond the harness/meta-harness abstraction, which is a modeling choice rather than an invented entity.

free parameters (4)

PosterBench rubric weights alpha = (10, 10, 15, 10, 20, 25, 10)
Hand-specified weights in Eq. (7) that define the overall score; all reported benchmark numbers depend on them.
Protected gate caps (e.g., P0 = 40) = 40 for P0, other gates not enumerated
Record-level ceilings in Eq. (8) applied before averaging; chosen by the authors and affecting all scores.
Refinement attempt budget K = 12
Maximum inner-loop repair attempts in Section 4.3; a design choice that affects output quality and cost.
Style-homogeneity check threshold = 20 posters
Minimum batch size for the blinded style-homogeneity adjustment in Appendix A.4; chosen by the authors and not applied to the 10-paper mini set.

assumptions (3)

domain assumption The frozen PosterBench evaluator is a valid measure of human poster quality.
The system is optimized toward Rmeta and evaluated with PosterBench, both built by the authors; the paper's only external validation is a human study with low inter-rater agreement (alpha = 0.101).
domain assumption Dtrain and Ddev are independent and Ddev is not part of the final Main Track evaluation.
Algorithm 1 gates updates on a development set, but the paper never states whether the 100-paper Main Track includes these papers; if they overlap, the headline result is not held out.
domain assumption Rmeta, initialized from human-annotated reference artifacts, provides a training signal that generalizes to the frozen evaluation.
The optimization loop relies on Rmeta remaining aligned with final quality; no analysis of reward hacking or Rmeta/PosterBench disagreement is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design." pith.science (2026). https://pith.science/paper/DWZGCBZY

@misc{pith2026260813560,
  author       = {Pith},
  title        = {Pith review of: AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DWZGCBZY}},
  note         = {Machine review of arXiv:2608.13560}
}

abstract

Transforming multimodal sources into condensed and structured media outputs can be fundamentally conceptualized as a long-horizon agentic process centered on a model-harness system. While an ideal harness system should align with human design priors and accumulate reusable experience through empirical exploration to drive recursive self-improvement, existing paradigms remain static and fall short of this capability. In this paper, we present AutoDesign, a framework that aligns with human design priors, where a meta-harness optimizer guides a code agent to recursively improve harness based on rollout feedback. To instantiate and evaluate this framework, we focus on the academic paper-to-poster generation task and introduce PosterBench, comprising a 100-paper Main Track spanning five disciplines and PosterBench-mini, a shared 10-paper subset for controlled evaluation. On the PosterBench Main Track, AutoDesign achieves the highest score of 78.32, surpassing the closed-source commercial system Claude Design by 7.45 points. Across seven controlled code-agent-model configurations, integrating the learned DesignHarness consistently improves performance, increasing the average PosterBench Score from 54.99 to 67.39 (+12.4%). In a fully autonomous long-horizon loop, it executes 253 tool calls and 11 editing turns within 40 minutes for under $3, reaching average conference-poster quality in human evaluation. A system-blind human study further demonstrates that AutoDesign achieves the highest human preference among evaluated systems.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 6 canonical work pages

[1]

Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J

Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA: Reflective prompt evolution can outperform reinforcement learning. InThe Fourteenth In...

work page 2026
[5]

Tsu-Jui Fu, William Yang Wang, Daniel McDuff, and Yale Song

doi: 10.18653/v1/2026.acl-long.15.https://aclanthology.org/2026.acl-long.15/. Tsu-Jui Fu, William Yang Wang, Daniel McDuff, and Yale Song. DOC2PPT: Automatic presentation slides generation from scientific documents. InAAAI, 2022.https://arxiv.org/abs/2101.11796. Jiaxin Ge, Zora Zhiruo Wang, Xuhui Zhou, Yi-Hao Peng, Sanjay Subramanian, Qinyue Tan, Maarten ...

arXiv 2026
[7]

Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T

https://arxiv.org/abs/2605.09998. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy: Compiling declarative language model calls into state-of-the-art pipelines. InThe Twelfth International ...

arXiv 2024
[11]

Design2Code: Benchmarking multimodal code generation for automated front-end engineering

Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. Design2Code: Benchmarking multimodal code generation for automated front-end engineering. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3956...

work page 2025
[12]

Tao Sun, Enhao Pan, Zhengkai Yang, Kaixin Sui, Jiajun Shi, Xianfu Cheng, Tongliang Li, Wenhao Huang, Ge Zhang, Jian Yang, and Zhoujun Li

doi: 10.18653/v1/ 2025.naacl-long.199.https://aclanthology.org/2025.naacl-long.199/. Tao Sun, Enhao Pan, Zhengkai Yang, Kaixin Sui, Jiajun Shi, Xianfu Cheng, Tongliang Li, Wenhao Huang, Ge Zhang, Jian Yang, and Zhoujun Li. P2P: Automated paper-to-poster generation and fine-grained benchmark. InInternational Conference on Learning Representations, 2026.htt...

arXiv 2025
[13]

SciPostLayout: A dataset for layout analysis and layout generation of scientific posters

Hao Wang, Shohei Tanaka, and Yoshitaka Ushiku. SciPostLayout: A dataset for layout analysis and layout generation of scientific posters. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 18 Workshops, pages 8136–8141, 2024.https://openaccess.thecvf.com/content/CVPR2024W/GDUG/html/Wang_ SciPostLayout_A_Dataset_for_Layout_A...

arXiv 2024
[15]

Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua

doi: 10.18653/v1/2024.naacl-long.417.https://aclanthology.org/2024.naacl-long.417/. Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. NExT-GPT: Any-to-any multimodal LLM.arXiv preprint arXiv:2309.05519, 2023.https://arxiv.org/abs/2309.05519. Chunqiu Steven Xia, Zhe Wang, Yan Yang, Yuxiang Wei, and Lingming Zhang. Live-SWE-agent: Can software ...

arXiv 2024
[16]

differentiation

https://arxiv.org/abs/2509.11062. Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. TextGrad: Automatic “differentiation” via text.arXiv preprint arXiv:2406.07496, 2024.https://arxiv.org/abs/2406.07496. Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai. Self-taught optimizer (STOP): Recu...

arXiv 2024

Show all 18 references

[17]

Zeyu Zhu, Kevin Qinghong Lin, and Mike Zheng Shou

doi: 10.18653/v1/2025.emnlp-main.728.https://aclanthology.org/2025.emnlp-main.728/. Zeyu Zhu, Kevin Qinghong Lin, and Mike Zheng Shou. Paper2Video: Automatic video generation from scientific papers.arXiv preprint arXiv:2510.05096, 2025.https://arxiv.org/abs/2510.05096. Mingche...

2025
[18]

source-flow-unit figure-flow-unit

https://proceedings.mlr.press/v235/zhuge24a.html. Mingchen Zhuge, Changsheng Zhao, Dylan R. Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, Yangyang Shi, Vikas Chandra, and Jürgen Schmidhuber. Agent-as-a...

2025
[100]

case_id":

A standard P0 gate has a ceiling of 40, while more severe gate types may impose a lower ceiling. Thus, ifq = N −1 P i qi is the vector displayed by a table row, 1 10 α⊤q does not generally equalOverall: the ceiling is applied before the benchmark average. For batches of at lea...

2017
[600]

0/600” counter denotes this complete roster; submitted non-skip decisions are retained and uncompleted assignments are not imputed. Human review decision record (schema excerpt) {

It evaluates each of the six unordered system pairs on every paper, providing equal paper coverage and a connected graph for Bradley–Terry estimation. The “0/600” counter denotes this complete roster; submitted non-skip decisions are retained and uncompleted assignments are no...

2017
[1987]

Goedel machines: Self-referential universal problem solvers making provably optimal self- improvements.arXiv preprint cs/0309048, 2006.https://arxiv.org/abs/cs/0309048

Jürgen Schmidhuber. Goedel machines: Self-referential universal problem solvers making provably optimal self- improvements.arXiv preprint cs/0309048, 2006.https://arxiv.org/abs/cs/0309048. Jürgen Schmidhuber, Jieyu Zhao, and Marco A. Wiering. Shifting inductive bias with succe...

2006 arXiv
[1995]

Jason Wu, Eldon Schoop, Alan Leung, Titus Barik, Jeffrey P

doi: 10.1017/S0269888900008122. Jason Wu, Eldon Schoop, Alan Leung, Titus Barik, Jeffrey P. Bigham, and Jeffrey Nichols. UICoder: Finetuning large language models to generate user interface code through automated feedback. InProceedings of the 2024 Conference of the North Amer...

doi
[1997]

Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao

doi: 10.1023/A: 1007383707642. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning.Advances in neural information processing systems, 36:8634–8652,

doi
[2023]

LongCat-Next: Lexicalizing modalities as discrete tokens.arXiv preprint arXiv:2603.27538, 2026.https://arxiv.org/abs/2603.27538

Meituan LongCat Team, Bin Xiao, Chao Wang, Chengjiang Li, Chi Zhang, Chong Peng, Hang Yu, Hao Yang, Haonan Yan, Haoze Sun, et al. LongCat-Next: Lexicalizing modalities as discrete tokens.arXiv preprint arXiv:2603.27538, 2026.https://arxiv.org/abs/2603.27538. Michael Nguyen, Qu...

2026
[2024]

Tingyang Chen, Shuo Lu, Kang Zhao, Weicheng Meng, Hanlin Teng, Tianhao Li, Chao Li, Xule Liu, Jian Liang, Zhizhong Zhang, Yuan Xie, Heng Qu, Kun Shao, and Jian Luan

https://arxiv.org/abs/2405.09818. Tingyang Chen, Shuo Lu, Kang Zhao, Weicheng Meng, Hanlin Teng, Tianhao Li, Chao Li, Xule Liu, Jian Liang, Zhizhong Zhang, Yuan Xie, Heng Qu, Kun Shao, and Jian Luan. HarnessX: A composable, adaptive, and evolvable agent harness foundry.arXiv p...

2026 arXiv
[2026]

Chameleon Team

https://arxiv.org/abs/2605.22794. Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818,

arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J

Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA: Reflective prompt evolution can outperform reinforcement learning. InThe Fourteenth In...

work page 2026

[2] [5]

Tsu-Jui Fu, William Yang Wang, Daniel McDuff, and Yale Song

doi: 10.18653/v1/2026.acl-long.15.https://aclanthology.org/2026.acl-long.15/. Tsu-Jui Fu, William Yang Wang, Daniel McDuff, and Yale Song. DOC2PPT: Automatic presentation slides generation from scientific documents. InAAAI, 2022.https://arxiv.org/abs/2101.11796. Jiaxin Ge, Zora Zhiruo Wang, Xuhui Zhou, Yi-Hao Peng, Sanjay Subramanian, Qinyue Tan, Maarten ...

arXiv 2026

[3] [7]

Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T

https://arxiv.org/abs/2605.09998. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy: Compiling declarative language model calls into state-of-the-art pipelines. InThe Twelfth International ...

arXiv 2024

[4] [11]

Design2Code: Benchmarking multimodal code generation for automated front-end engineering

Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. Design2Code: Benchmarking multimodal code generation for automated front-end engineering. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3956...

work page 2025

[5] [12]

Tao Sun, Enhao Pan, Zhengkai Yang, Kaixin Sui, Jiajun Shi, Xianfu Cheng, Tongliang Li, Wenhao Huang, Ge Zhang, Jian Yang, and Zhoujun Li

doi: 10.18653/v1/ 2025.naacl-long.199.https://aclanthology.org/2025.naacl-long.199/. Tao Sun, Enhao Pan, Zhengkai Yang, Kaixin Sui, Jiajun Shi, Xianfu Cheng, Tongliang Li, Wenhao Huang, Ge Zhang, Jian Yang, and Zhoujun Li. P2P: Automated paper-to-poster generation and fine-grained benchmark. InInternational Conference on Learning Representations, 2026.htt...

arXiv 2025

[6] [13]

SciPostLayout: A dataset for layout analysis and layout generation of scientific posters

Hao Wang, Shohei Tanaka, and Yoshitaka Ushiku. SciPostLayout: A dataset for layout analysis and layout generation of scientific posters. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 18 Workshops, pages 8136–8141, 2024.https://openaccess.thecvf.com/content/CVPR2024W/GDUG/html/Wang_ SciPostLayout_A_Dataset_for_Layout_A...

arXiv 2024

[7] [15]

Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua

doi: 10.18653/v1/2024.naacl-long.417.https://aclanthology.org/2024.naacl-long.417/. Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. NExT-GPT: Any-to-any multimodal LLM.arXiv preprint arXiv:2309.05519, 2023.https://arxiv.org/abs/2309.05519. Chunqiu Steven Xia, Zhe Wang, Yan Yang, Yuxiang Wei, and Lingming Zhang. Live-SWE-agent: Can software ...

arXiv 2024

[8] [16]

differentiation

https://arxiv.org/abs/2509.11062. Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. TextGrad: Automatic “differentiation” via text.arXiv preprint arXiv:2406.07496, 2024.https://arxiv.org/abs/2406.07496. Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai. Self-taught optimizer (STOP): Recu...

arXiv 2024