Pith. sign in

REVIEW 3 major objections 8 minor 1 cited by

RepoTransAgent: Multi-Agent LLM Framework for Repository-Aware Code Translation

T0 review · 3 major / 8 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A three-agent LLM framework raises repository-aware Java-C# translation to 55.34% compile and 45.84% pass rates.

desk verdict A believable multi-agent code-translation framework with a real counting inconsistency in the reported evaluation; the headline numbers need artifact release and a denominator fix before they can be trusted. read the letter →

arxiv 2508.17720 v1 pith:TUG5GDRO submitted 2025-08-25 cs.SE

Ziqi Guan , Xin Yin , Zhiyuan Peng , Chao Ni This is my paper · ORCID
classification cs.SE
keywords multi-agentLLMframeworkrepository-awarecodetranslationretrieval-augmentedgenerationiterativerefinementJava-C#reflection-basederrorcorrectionagents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that repository-aware code translation, where a function is translated inside a real codebase rather than in isolation, fails with static prompts and plain test feedback, and that decomposing the job into three collaborating LLM agents fixes much of the failure. The agents retrieve similar functions, gather repository context through five selectable tools, and iteratively translate, run tests, reflect on failures, and re-fetch context before fixing. On hundreds of Java-C# function pairs from six open-source projects, the framework reports compile rates up to 55.34% and pass rates up to 45.84%, roughly doubling the baselines' 26-30% compile and 18-28% pass rates. If this holds, it suggests that copying code across ecosystems is mostly a context problem, finding the target repository's substitutes for classes, methods, and imports, and that agent-driven context retrieval is a practical way to solve it.

What carries the argument

The load-bearing object is the three-agent workflow with a dynamic prompt cycle. The RAG Agent decides whether retrieval is needed and, when it is, pulls similar source-target pairs and similarly named functions from vector stores. The Context Agent repeatedly chooses among five tools, get source class info, get target class info, find target imports, find target class info, and find target method body, each returning JSON, until enough context is gathered. The Refine Agent generates the translation, runs the target project's tests, classifies failures into compilation, runtime, functional, or non-terminating errors, reflects on root causes, re-invokes context tools when needed, and produces a corrected version. Static prompt components such as Goals, Guidelines, Example, and Output Format are coupled with dynamic components such as Input, Gathered Context, and Last Command, which is what lets the agent adapt its tool use to the specific function and prior results.

What would settle it

A concrete check: take a random sample of the dataset's translation pairs, have developers verify each Java-C# pair is semantically equivalent and that the unit tests exercise the translated method, then re-run the framework on the verified subset. If the compile and pass rates fall well below the reported 55.34% and 45.84%, the headline numbers are an artifact of pair alignment or test selection.

Watch

Extended reading notes

Core claim

RepoTransAgent's central claim is that a multi-agent decomposition, with a RAG Agent that retrieves similar functions, a Context Agent that invokes tools to pull class and method information from the target repository, and a Refine Agent that translates, executes tests, reflects on root causes, and re-retrieves context before correcting, materially improves repository-aware code translation over prior LLM translation pipelines. The paper reports that on its Java-C# dataset, the framework reaches an average compile rate of 55.34% for C# to Java and 43.07% for Java to C#, with pass rates of 45.84% and 32.36%, against baseline compile rates of 26.07% to 30.47% and pass rates of 18.59% to 28.16%. Ablation results attribute most of the gain to the Context and Refine agents, with the RAG Agent contributing a smaller but positive effect. The authors also report that the framework improves all tested LLMs, with larger models benefiting more and small models sometimes degrading, and that different models translate complementary sets of functions.

Load-bearing premise

The whole comparison rests on the assumption that the Java-C# function pairs taken from six open-source projects are correctly aligned and that each project's executed unit tests genuinely test the translated function's behavior; if either the pairing or the tests are wrong, the reported compile and pass rates do not measure translation correctness.

Editorial extensions

If this is right

  • Because the dominant failure is the 'symbol not found' error, translation success depends on locating the target repository's equivalent classes and methods, so the same agent-tool pattern is a plausible template for other language pairs.
  • The ablation's largest drop when the Refine Agent is removed implies that iterative test feedback alone is not enough; root-cause reflection plus targeted context re-retrieval is what drives the gain.
  • The model-size result implies that agentic translation benefits are only accessible above a capability threshold, and smaller models may need simplified prompts to participate.
  • The intersection analysis implies that different models fail on different functions, so combining model outputs yields more correct translations than any single model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit is applying the same retrieve-context-then-reflect loop to API migration and framework upgrades, where the hard step is also substituting target-ecosystem equivalents rather than rewriting syntax.
  • The small RAG contribution suggests a testable tweak: loosen the standalone-function gate that skips retrieval and measure whether more similar-function examples help the hard cases.
  • The dependence on prompt length suggests a testable simplification: compressed, tool-specific prompts might bring the benefit to smaller 7B/8B models, where the paper observes degradation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. RepoTransAgent is a multi-agent LLM framework for repository-aware Java-C# code translation. The system decomposes translation into three specialized roles: a RAG agent that retrieves similar source-target pairs and name matches, a context agent that invokes five repository tools to gather class, import, and method information, and a refine agent that generates code, executes tests, reflects on failures, and iteratively corrects errors, all driven by a dynamic prompting scheme. The authors construct a dataset from six dual-language GitHub projects (Table I: 627 Java and 655 C# focal methods), claim to follow Methods2Test, and evaluate against UniTrans, PLTranslation, and a no-agent baseline using compile and pass rates. With DeepSeek V3 on the C#-to-Java direction they report 363 compiled and 299 passed out of 627 tasks (55.34% average compile, 45.84% average pass), and 306 compiled and 245 passed out of 655 tasks for Java-to-C#. Ablations (Table IV) show the context and refine agents matter most, and cross-model experiments (Figs. 6-8) show consistent gains on larger models but smaller gains on 7B/8B models. The paper claims up to 55.34% compile rate and 45.84% pass rate and asserts significant outperformance of the baselines.

Significance. The paper offers a useful engineering-level integration of retrieval-augmented generation, tool-based context gathering, and reflection-based iterative refinement in a multi-agent pipeline, and the design is internally well motivated. The strengths are the clean ablation isolating each agent's contribution (Table IV), the breadth of the cross-model evaluation over six LLMs at two size scales, a concrete case study showing tool-driven repair grounded in repository context (Fig. 4), and an honest threats section that checks for data leakage and reports agent temperatures. If the reported rates survive a corrected and released dataset, the framework would be a practical advance for repository-aware translation. However, the significance is currently conditional: the evaluation rests entirely on the authors' own unreleased six-project dataset, the reported task counts are internally inconsistent with the dataset statistics, there are no statistical tests or repeated runs, and no external repository-level benchmark anchors the claimed state-of-the-art superiority. The claimed improvements are plausible but not yet established at the level the abstract asserts.

major comments (3)
  1. [V-A(a), Tables I-III] The reported translation counts are internally inconsistent with the dataset statistics in Table I. The focal-method column sums are 113+229+134+67+42+42 = 627 Java methods and 114+291+140+35+25+50 = 655 C# methods, yet Section V-A(a) reports 627 functions translated from C# to Java and 655 functions translated from Java to C#. For a paired dataset, a Java-to-C# task requires a Java source, so the direction with 655 tasks cannot be produced from only 627 Java focal methods, and conversely the C#-to-Java direction should not have exactly the Java focal-method count unless sources are defined differently. The per-project percentages in Table II use these same counts as denominators (for example, lucene 49.56% times 113 and 15.79% times 114), and Table III's arithmetic (627-363=264 and 655-306=349) confirms that the 627/655 totals are not isolated typos. Because compile rate and pass rate are defined as proportions of these tasks, and the abstract's headline 55.34% and 45.84% figures are computed from these denominators, the central quantitative claims are not verifiable as reported until the authors specify the exact pairing and alignment algorithm, justify the per-direction counts, and recompute all rates on corrected denominators.
  2. [IV-A, III-B3, III-C1, III-D4] The dataset construction and the evaluation oracle are under-specified to the point of irreproducibility. Saying that the dataset follows the established methodology of Methods2Test cannot be literally valid, because Methods2Test aligns Java focal methods with their Java test methods and provides no cross-language pairing procedure; the paper never states how the Java and C# implementations of each project were aligned (by names, signatures, or tests), why the per-project focal counts differ between languages in Table I, or whether each translation pair has both a Java and a C# side. It is also unclear what is executed when computing pass rate: whether the target repository's existing tests, the source-repository tests ported to the target language, or both. In addition, no dataset or code is released, and key hyperparameters are never given: the retrieval top-k for the RAG agent, the maximum tool-invocation iterations for the context agent, and the maximum refinement iterations for the refine agent. These omissions place the entire baseline comparison beyond independent verification.
  3. [V (RQ-1 to RQ-3), VI] The claim that RepoTransAgent significantly outperforms the baselines is not supported by the statistical evidence presented. Because the Refine Agent runs at temperature 0.8 to encourage diverse hypotheses (Section VI), outputs are stochastic, yet every rate in Tables II-IV and Figs. 6-8 comes from a single run with no confidence intervals, no repeated-seed experiments, and no significance tests; with six projects the paper does not even perform a paired comparison or report per-project baseline deltas in a way that would support a claim of significance. The Average rows in Table II are also unweighted means of per-project rates, with 55.34% being the mean of the six project-level compile rates rather than the pooled rate of 363/627 = 57.9%, and this convention is never stated. Finally, Section V-C reports that on Llama3.1-8B and Qwen2.5-7B the improvements were considerably smaller and even exhibited decline in some cases (Fig. 7), which materially qualifies the model-agnostic generalizability claim in the abstract; without variance information the robustness conclusions cannot be assessed.
minor comments (8)
  1. [IV] The first paragraph of Section IV reads 'methodology employ to evaluate' where 'methodology employed' is intended, and similar grammar slips appear elsewhere (for example, Section III-D1 'configure with specialized expertise'), so the manuscript needs a careful copyedit.
  2. [Table II] The Average rows should state explicitly that they are unweighted means over the six projects; as computed they differ from the pooled rates (for example, 363/627 = 57.9% versus 55.34% for RepoTransAgent's C#-to-Java compile rate), and the current labeling invites misinterpretation.
  3. [Abstract] The abstract's phrasing that the framework 'achieves up to 55.34% compile rate and 45.84% pass rate' presents average rates as if they were upper bounds; the per-project maxima are substantially higher (for example, 73.81% compile rate on quartz), so 'up to' is misleading.
  4. [VII-A] Rectifier [13], the authors' own prior translation system, is cited but never used as a baseline; given the stated goal of outperforming state-of-the-art translation methods, the omission should be justified, for example by noting explicitly that Rectifier does not handle repository context.
  5. [Figs. 6, 7] Figures 6 and 7 are nearly unreadable because the fonts are small and the value labels are crowded and overlapping (for example, '20.4 20.5'); a cleaner layout with error bars would also mitigate the single-run reporting issue.
  6. [VI] The data-leakage check based on exact matching after removing whitespace and comments is weak, because a model that memorized a function could still emit it with renamed variables or reformatting; a stricter similarity metric or a held-out verification would be more convincing.
  7. [Figs. 5, 8] The set-intersection analyses (90 additional translations, 104 common, 355 union) report absolute counts without stating the underlying task totals or whether these are single-run observations, which limits their interpretability.
  8. [Table I] Table I shows a wide imbalance in per-project focal counts (for example, itext 67/35 and quartz 42/25); since the headline averages are unweighted, the smallest projects have the same influence as poi (229/291), and this should be acknowledged when interpreting the aggregate numbers.

Circularity Check

1 steps flagged · score 6.0 of 10

RAG pair store can feed the ground-truth target back as context, so the claimed translation 'prediction' reduces to retrieving the answer from the input.

  1. self definitional [Section III-B1 (Preprocessing), III-B3 (Multi-Route Retrieval), and III-B4 (Result Filtering)]
    "The first component consists of the method bodies of all source-target translation pairs... The first component involves retrieving source-target translation pairs where the source function is similar to the current source function from the pair store. ... For each candidate function in the retrieved set, the agent compares it with the target function."

    In a translation run, the current source function is itself one of the stored source-target translation pairs, so its ground-truth target is a candidate in the pair store with maximal self-similarity. The paper states no exclusion of the current pair from the vector database or from the retrieved top-k. The Result Filtering step then instructs the agent to keep candidates that are functionally similar to the target function, which the ground-truth pair trivially satisfies. The RAG context supplied to the Refine Agent therefore can contain the exact expected translation; the reported compile and pass rates are then not a prediction derived from repository context but a re-presentation of an input. This makes the central evaluation result circular by construction for the RAG-augmented runs.

full rationale

RepoTransAgent is an empirical systems paper, so most of its claimed improvements rest on test-suite execution rather than on a deductive derivation; the usual circularity patterns such as self-citation chains, imported uniqueness theorems, or ansatz-smuggling do not appear. The baseline comparison and ablation involve no fitted parameters that are later renamed as predictions, and the self-cited Rectifier paper is not used as a load-bearing baseline or justification. The one serious circularity is the RAG pair store: because it indexes the method bodies of all source-target translation pairs and retrieval is triggered by the current source function, the ground-truth target for that very function can be fed back as similar-function context, with no described self-exclusion. This can make the headline compile and pass rates an artifact of retrieving the answer rather than translating. In addition, Section V-A's directional counts (627 vs. 655) are inconsistent with Table I's focal-method sums and with the stated direction labels, and the dataset alignment under Methods2Test is under-described; these are validity threats but not derivation-level circularity. The evaluation is entirely on the authors' own six-project dataset, so the RAG leakage concern directly affects the central claim. The score of 6 reflects partial circularity: the framework's RAG input can contain the output, but the Refine Agent must still generate code and pass tests, so the reduction is not absolute.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No theoretical entities are introduced; the framework is an engineered system. The main epistemic burden is the self-constructed dataset and the unshipped implementation.

free parameters (4)
  • Refine Agent temperature = 0.8
    Chosen to encourage diverse hypotheses during iterative repair; directly affects stochastic output and measured rates.
  • RAG and Context Agent temperature = 0
    Set to 0 to reduce randomness; affects retrieval and context selection.
  • Retrieval top-k = not reported
    Number of similar functions or names retrieved in multi-route retrieval and RRF fusion is not specified.
  • Maximum refinement iterations = not reported
    Termination condition for iterative correction; not specified.
assumptions (4)
  • domain assumption The six selected GitHub projects contain valid, aligned Java-C# function pairs with corresponding test cases.
    All evaluation depends on the correctness of the self-constructed dataset (Section IV-A).
  • domain assumption Passing the associated unit tests is a sufficient oracle for translation correctness.
    Pass rate is defined as 'correct' if all associated test cases pass (Section IV-C); weak or incomplete tests would overstate correctness.
  • domain assumption Baseline prompts adapted by the authors are fair representatives of UniTrans and PLTranslation.
    Baselines are not released; fairness of adaptation cannot be checked (Section IV-B).
  • domain assumption Low exact-match rate after normalization indicates low data leakage.
    The internal validity check assumes exact match with ground truth after removing whitespace or comments is a meaningful leakage signal (Section VI).

how reviews work

0 comments
Cite this review

Pith. "Pith review of RepoTransAgent: Multi-Agent LLM Framework for Repository-Aware Code Translation." pith.science (2026). https://pith.science/paper/TUG5GDRO

@misc{pith2026250817720,
  author       = {Pith},
  title        = {Pith review of: RepoTransAgent: Multi-Agent LLM Framework for Repository-Aware Code Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TUG5GDRO}},
  note         = {Machine review of arXiv:2508.17720}
}
read the original abstract

Repository-aware code translation is critical for modernizing legacy systems, enhancing maintainability, and enabling interoperability across diverse programming languages. While recent advances in large language models (LLMs) have improved code translation quality, existing approaches face significant challenges in practical scenarios: insufficient contextual understanding, inflexible prompt designs, and inadequate error correction mechanisms. These limitations severely hinder accurate and efficient translation of complex, real-world code repositories. To address these challenges, we propose RepoTransAgent, a novel multi-agent LLM framework for repository-aware code translation. RepoTransAgent systematically decomposes the translation process into specialized subtasks-context retrieval, dynamic prompt construction, and iterative code refinement-each handled by dedicated agents. Our approach leverages retrieval-augmented generation (RAG) for contextual information gathering, employs adaptive prompts tailored to varying repository scenarios, and introduces a reflection-based mechanism for systematic error correction. We evaluate RepoTransAgent on hundreds of Java-C# translation pairs from six popular open-source projects. Experimental results demonstrate that RepoTransAgent significantly outperforms state-of-the-art baselines in both compile and pass rates. Specifically, RepoTransAgent achieves up to 55.34% compile rate and 45.84% pass rate. Comprehensive analysis confirms the robustness and generalizability of RepoTransAgent across different LLMs, establishing its effectiveness for real-world repository-aware code translation.

Figures

Figures reproduced from arXiv: 2508.17720 by the authors.

Figure 1
Figure 1. An example of repository-aware code translation using static analysis methods specialized agents that collaborate systematically, our approach enhances the effectiveness, maintainability, and robustness of repository-aware code translation. Key Idea 2: Intelligent Context Retrieval via Tool Invo￾cation. Within the Context Agent, we design five distinct tool functions that enable selective invocation based on the age… view at source ↗
Figure 2
Figure 2. Overview of RepoTransAgent [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An example of the dynamic prompt construction process edly use find target class info and find target method body to gather sufficient context, including identifying new type dependencies, verifying interface implementations, and resolv￾ing mappings from packages to namespaces. Additionally, the agent is reminded to pay close attention to language-specific differences. d) Example (Static): We provide an example that… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: An example of a function translation using tools by RepoTransAgent c) Comparison with Baselines: When compared with baseline approaches, RepoTransAgent achieves substantially higher compile and pass rates across both translation directions and all projects, although th…
Figure 5
Figure 5. Figure 5: illustrates the overlap of successfully translated functions among different methods [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: RepoTransAgent vs. Baselines across different LLMs on average compile rate and average pass rate LLAMA3.1:8B LLAMA3.1:70B QWEN2.5:7B QWEN2.5:72B 0 10 20 30 40 Compile Rate (%) 15.3 20.5 13.8 20.4 15.0 30.8 12.4 35.5 10.3 22.7 14.0 27.8 10.9 22.5 12.5 24.3 Compile Rate(…
Figure 7
Figure 7. Figure 7: Average compile rate and average pass rate of LLMs with different sizes Answer to RQ-2: The ablation study demonstrates that the Context Agent and Refine Agent play crucial roles in the performance of RepoTransAgent, as the absence of either agent results in a substant…
Figure 8
Figure 8. Figure 8: The intersection of functions successfully translated by different LLMs Answer to RQ-3: RepoTransAgent substantially enhances the performance of base LLMs on repository-aware code translation tasks through its multi-agent framework. Addi￾tionally, the functions success…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Julia for CFD: A Critical Survey of Ecosystem, Performance, and Composability

    cs.CE 2026-08 conditional novelty 3.0 of 10

    Julia is now credible for several research-facing CFD regimes, but its advantage is integration and composability, not unique performance, and it is not yet a complete industrial CFD platform.

Reference graph

Works this paper leans on

38 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    Lexical statistical machine translation for language migration,

    A. T. Nguyen, T. T. Nguyen, and T. N. Nguyen, “Lexical statistical machine translation for language migration,” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering , 2013, pp. 651–654

    work page 2013

  2. [2]

    Tree-to-tree neural networks for program translation,

    X. Chen, C. Liu, and D. Song, “Tree-to-tree neural networks for program translation,” Advances in neural information processing systems , vol. 31, 2018

    work page 2018

  3. [3]

    Language to logical form with neural attention,

    L. Dong and M. Lapata, “Language to logical form with neural attention,” arXiv preprint arXiv:1601.01280 , 2016

    arXiv 2016

  4. [4]

    A syntactic neural model for general-purpose code generation,

    P. Yin and G. Neubig, “A syntactic neural model for general-purpose code generation,” arXiv preprint arXiv:1704.01696 , 2017

    arXiv 2017

  5. [5]

    Tranx: A transition-based neural abstract syntax parser for semantic parsing and code generation,

    ——, “Tranx: A transition-based neural abstract syntax parser for semantic parsing and code generation,” arXiv preprint arXiv:1810.02720, 2018

    arXiv 2018

  6. [6]

    Xcodeeval: An execution-based large scale multilingual multitask benchmark for code understanding, generation, translation and retrieval,

    M. A. M. Khan, M. S. Bari, D. Long, W. Wang, M. R. Parvez, and S. Joty, “Xcodeeval: An execution-based large scale multilingual multitask benchmark for code understanding, generation, translation and retrieval,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2024, pp. 6766– 6805

    work page 2024

  7. [7]

    Exploring the impact of the output format on the evaluation of large language models for code translation,

    M. Macedo, Y . Tian, F. Cogo, and B. Adams, “Exploring the impact of the output format on the evaluation of large language models for code translation,” in Proceedings of the 2024 IEEE/ACM First International Conference on AI Foundation Models and Software Engineering , 2024, pp. 57–68

    work page 2024

  8. [8]

    Lost in translation: A study of bugs introduced by large language models while translating code,

    R. Pan, A. R. Ibrahimzada, R. Krishna, D. Sankar, L. P. Wassi, M. Merler, B. Sobolev, R. Pavuluri, S. Sinha, and R. Jabbarvand, “Lost in translation: A study of bugs introduced by large language models while translating code,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering , 2024, pp. 1–13

    work page 2024

Show all 38 references
  1. [9]

    Exploring and unleashing the power of large language models in automated code translation,

    Z. Yang, F. Liu, Z. Yu, J. W. Keung, J. Li, S. Liu, Y . Hong, X. Ma, Z. Jin, and G. Li, “Exploring and unleashing the power of large language models in automated code translation,” Proceedings of the ACM on Software Engineering, vol. 1, no. FSE, pp. 1585–1608, 2024

    2024

  2. [10]

    Using model-driven engineering to automate software language translation,

    K. Lano and H. Siala, “Using model-driven engineering to automate software language translation,” Automated Software Engineering, vol. 31, no. 1, p. 20, 2024

    2024

  3. [11]

    Spectra: Enhancing the code translation ability of language models by generating multi-modal specifications,

    V . Nitin, R. Krishna, and B. Ray, “Spectra: Enhancing the code translation ability of language models by generating multi-modal specifications,” arXiv preprint arXiv:2405.18574 , 2024

    2024

  4. [12]

    Understanding the effectiveness of large language models in code translation. preprint (2023),

    R. Pan, A. Ibrahimzada, R. Krishna, D. Sankar, L. Wassi, M. Merler, B. Sobolev, R. Pavuluri, S. Sinha, and R. Jabbarvand, “Understanding the effectiveness of large language models in code translation. preprint (2023),” arXiv preprint arXiv:2308.03109

    2023 arXiv

  5. [13]

    Rectifier: Code translation with corrector via llms,

    X. Yin, C. Ni, T. N. Nguyen, S. Wang, and X. Yang, “Rectifier: Code translation with corrector via llms,” arXiv preprint arXiv:2407.07472 , 2024

    2024 arXiv

  6. [14]

    Repository-level code translation benchmark targeting rust,

    G. Ou, M. Liu, Y . Chen, X. Peng, and Z. Zheng, “Repository-level code translation benchmark targeting rust,” arXiv preprint arXiv:2411.13990 , 2024

    2024

  7. [15]

    Repotransbench: A real-world benchmark for repository-level code translation,

    Y . Wang, Y . Wang, S. Wang, D. Guo, J. Chen, J. Grundy, X. Liu, Y . Ma, M. Mao, H. Zhang, and Z. Zheng, “Repotransbench: A real-world benchmark for repository-level code translation,” 2024. [Online]. Available: https://arxiv.org/abs/2412.17744

    2024

  8. [16]

    Meth- ods2test: A dataset of focal methods mapped to test cases,

    M. Tufano, S. K. Deng, N. Sundaresan, and A. Svyatkovskiy, “Meth- ods2test: A dataset of focal methods mapped to test cases,” in Proceedings of the 19th International Conference on Mining Software Repositories , 2022, pp. 299–303

    2022

  9. [17]

    The llama 3 herd of models,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan et al., “The llama 3 herd of models,” arXiv preprint arXiv:2407.21783 , 2024

    2024 arXiv

  10. [18]

    Qwen3 technical report,

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv et al. , “Qwen3 technical report,” arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  11. [19]

    Gpt-4o system card,

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford et al., “Gpt-4o system card,” arXiv preprint arXiv:2410.21276 , 2024

    2024 arXiv

  12. [20]

    Deepseek-v3 technical report,

    A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan et al., “Deepseek-v3 technical report,” arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  13. [21]

    Codexglue: A machine learning benchmark dataset for code understanding and generation,

    S. Lu, D. Guo, S. Ren, J. Huang, A. Svyatkovskiy, A. Blanco, C. Clement, D. Drain, D. Jiang, D. Tang et al. , “Codexglue: A machine learning benchmark dataset for code understanding and generation,” arXiv preprint arXiv:2102.04664, 2021

    2021 arXiv

  14. [22]

    Codenet: A large- scale ai for code dataset for learning a diversity of coding tasks,

    R. Puri, D. S. Kung, G. Janssen, W. Zhang, G. Domeniconi, V . Zolotov, J. Dolby, J. Chen, M. Choudhury, L. Decker et al., “Codenet: A large- scale ai for code dataset for learning a diversity of coding tasks,” arXiv preprint arXiv:2105.12655, 2021

    2021 arXiv

  15. [23]

    Xlcost: A benchmark dataset for cross-lingual code intelligence,

    M. Zhu, A. Jain, K. Suresh, R. Ravindran, S. Tipirneni, and C. K. Reddy, “Xlcost: A benchmark dataset for cross-lingual code intelligence,” arXiv preprint arXiv:2206.08474, 2022

    2022 arXiv

  16. [24]

    Xtest: A parallel multilingual corpus with test cases for code translation and its evaluation,

    I. J. Rithy, H. H. Shakil, N. Mondal, F. Sultana, and F. M. Shah, “Xtest: A parallel multilingual corpus with test cases for code translation and its evaluation,” in 2022 25th International Conference on Computer and Information Technology (ICCIT). IEEE, 2022, pp. 623–628

    2022

  17. [25]

    On the evaluation of neural code translation: Taxonomy and benchmark,

    M. Jiao, T. Yu, X. Li, G. Qiu, X. Gu, and B. Shen, “On the evaluation of neural code translation: Taxonomy and benchmark,” in 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2023, pp. 1529–1541

    2023

  18. [26]

    Mindagent: Emergent gaming interaction,

    R. Gong, Q. Huang, X. Ma, H. V o, Z. Durante, Y . Noda, Z. Zheng, S.-C. Zhu, D. Terzopoulos, L. Fei-Fei et al., “Mindagent: Emergent gaming interaction,” arXiv preprint arXiv:2309.09971 , 2023

    2023 arXiv

  19. [27]

    Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents,

    W. Chen, Y . Su, J. Zuo, C. Yang, C. Yuan, C. Qian, C.-M. Chan, Y . Qin, Y . Lu, R. Xie et al. , “Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents,” arXiv preprint arXiv:2308.10848, vol. 2, no. 4, p. 6, 2023

    2023 arXiv

  20. [28]

    Building cooperative embodied agents modularly with large language models,

    H. Zhang, W. Du, J. Shan, Q. Zhou, Y . Du, J. B. Tenenbaum, T. Shu, and C. Gan, “Building cooperative embodied agents modularly with large language models,” arXiv preprint arXiv:2307.02485 , 2023

    2023 arXiv

  21. [29]

    Metagpt: Meta programming for multi-agent collaborative framework,

    S. Hong, X. Zheng, J. Chen, Y . Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou et al., “Metagpt: Meta programming for multi-agent collaborative framework,” arXiv preprint arXiv:2308.00352 , vol. 3, no. 4, p. 6, 2023

    2023 arXiv

  22. [30]

    Proagent: building proactive cooperative agents with large language models,

    C. Zhang, K. Yang, S. Hu, Z. Wang, G. Li, Y . Sun, C. Zhang, Z. Zhang, A. Liu, S.-C. Zhu et al., “Proagent: building proactive cooperative agents with large language models,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 16, 2024, pp. 17 591–17 599

    2024

  23. [31]

    Llmarena: Assessing capabilities of large language models in dynamic multi-agent environments,

    J. Chen, X. Hu, S. Liu, S. Huang, W.-W. Tu, Z. He, and L. Wen, “Llmarena: Assessing capabilities of large language models in dynamic multi-agent environments,” arXiv preprint arXiv:2402.16499 , 2024

    2024 arXiv

  24. [32]

    Villageragent: A graph- based multi-agent framework for coordinating complex task dependencies in minecraft,

    Y . Dong, X. Zhu, Z. Pan, L. Zhu, and Y . Yang, “Villageragent: A graph- based multi-agent framework for coordinating complex task dependencies in minecraft,” arXiv preprint arXiv:2406.05720 , 2024

    2024 arXiv

  25. [33]

    Project sid: Many-agent simulations toward ai civilization,

    A. AL, A. Ahn, N. Becker, S. Carroll, N. Christie, M. Cortes, A. Demirci, M. Du, F. Li, S. Luo et al., “Project sid: Many-agent simulations toward ai civilization,” arXiv preprint arXiv:2411.00114 , 2024

    2024 arXiv

  26. [34]

    Evolution of innovative drug r&d in china,

    G. Li, Y . Liu, H. Hu, S. Yuan, L. Zhou, and X. Chen, “Evolution of innovative drug r&d in china,” Nat Rev Drug Discov , vol. 21, no. 8, pp. 553–554, 2022

    2022

  27. [35]

    Epidemiology and risk factors in osteoarthritis: literature review data from “oasis

    L. De Filippis, S. Gulli, A. Caliri, C. Romano, F. Muna `o, G. Trimarchi, D. La Torre, C. Fichera, A. Pappalardo, G. Triolo et al., “Epidemiology and risk factors in osteoarthritis: literature review data from “oasis” study,” Reumatismo, vol. 56, no. 3, pp. 169–184, 2004

    2004

  28. [36]

    Marg: Multi-agent review generation for scientific papers,

    M. D’Arcy, T. Hope, L. Birnbaum, and D. Downey, “Marg: Multi-agent review generation for scientific papers,” arXiv preprint arXiv:2401.04259, 2024

    2024 arXiv

  29. [37]

    Autocoderover: Autonomous program improvement,

    Y . Zhang, H. Ruan, Z. Fan, and A. Roychoudhury, “Autocoderover: Autonomous program improvement,” in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis , 2024, pp. 1592–1604

    2024

  30. [38]

    Swe-agent: Agent-computer interfaces enable automated software engineering,

    J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press, “Swe-agent: Agent-computer interfaces enable automated software engineering,” Advances in Neural Information Processing Systems, vol. 37, pp. 50 528–50 652, 2024

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.