Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

MatchFixAgent: Language-Agnostic Autonomous Repository-Level Code Translation Validation and Repair

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MatchFixAgent claims a language-agnostic LLM multi-agent system can validate and repair repository-level code translations, returning verdicts for 99.2% of 2,219 translation pairs and fixing 50.6% of detected bugs, versus 18.5% for prior…

desk verdict A credible, low-overhead multi-PL translation validation/repair system whose headline accuracy claim is not actually measured: 68.5% of verdicts (the EQ outcomes) are never checked against ground truth. read the letter →

arxiv 2509.16187 v3 pith:VLOPI4W7 submitted 2025-09-19 cs.SE cs.LG

Ali Reza Ibrahimzada , Brandon Paulsen , Reyhaneh Jabbarvand , Joey Dodds , Daniel Kroening This is my paper · ORCID
classification cs.SEcs.LG
keywords codetranslationfunctionalequivalencevalidationLLMagentsprogramanalysisautomatedrepairlanguage-agnosticrepository-levelmulti-agentsystem
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MatchFixAgent is a claim that code translation validation and repair can be made language-agnostic and autonomous, replacing the hand-built per-language-pair machinery of existing tools with a small LLM-based multi-agent pipeline. The paper argues that splitting equivalence analysis into six semantic views (control flow, data flow, input/output behavior, library APIs, exception handling, and specifications), feeding those reports to an agent that writes and runs tests and patches failing translations, and finally letting a verdict agent reconcile everything is enough to catch translation bugs that prior validation misses. If the paper is right, teams migrating whole repositories from one programming language to another can get near-universal equivalence verdicts at roughly 1,650 lines of code, about 280 lines per additional language, with repair rates more than double prior reprompting approaches. The central evidence is 2,219 source–translation pairs across six language pairs from twenty-four real projects: verdicts for 99.2%, agreement with prior tools on 72.8% of pairs where both sides produced verdicts, and 60.7% of sampled disagreements judged in MatchFixAgent's favor by manual review.

What carries the argument

The load-bearing mechanism is the division of labor among three components. The Semantic Analyzer runs six independent LLM sub-analyzers over control-flow graphs, data-flow paths, I/O mappings, library API usage, exception handling, and inferred specifications; two of those sub-analyzers first compute a Jaccard or edit-distance similarity score between abstracted graphs and skip the LLM when similarity exceeds 0.7. The Test Generator and Repair Agent takes the six reports plus a functional-equivalence definition and, using an off-the-shelf coding agent, writes and executes tests in both languages and iteratively patches the translation when tests fail. The Verdict Agent reconciles those reports into a final equivalence verdict. The claim is that this decomposition gives the LLM a focused, testable view of each semantic dimension, so that hallucinated differences are caught by test execution before a final verdict is issued.

What would settle it

Independently re-audit a random sample of translation pairs that MatchFixAgent labeled equivalent and that prior tools also labeled equivalent, using a formal equivalence checker or an independent expert panel; if a meaningful fraction turn out inequivalent, the claim that MatchFixAgent is more accurate than prior work loses its support.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that a multi-agent LLM pipeline, guided by cheap approximate semantic analyses, is a viable replacement for the heavy language-interoperability machinery used in prior repository-level translation validation. The system decomposes functional equivalence into six independent semantic dimensions, computes control-flow and data-flow graphs from the source and translation, uses graph-similarity thresholds to short-circuit obviously equivalent cases, hands the remaining analysis to an agent that generates and executes differential tests, attempts repairs on failing tests, and concludes with a verdict agent. On 2,219 translation pairs the pipeline returns an equivalence verdict for 99.2%, agrees with earlier validation tools on 72.8% of the 1,571 pairs where both sides produced verdicts, and in a manual audit of 145 disagreements was judged correct in 60.7%, with repair success of 50.6% of 265 buggy translations compared to 18.5% for prior approaches. The authors also report that removing the semantic analyzer and test generator drops validation accuracy by 42.3%, and that the whole system is only 1,650 lines of code with about 280 additional lines per supported language pair.

Load-bearing premise

The evaluation never checks whether translations judged equivalent are actually equivalent, so the claimed validation accuracy assumes the unexamined 'equivalent' verdicts are reliable.

Editorial extensions

If this is right

  • Repository-level translation validation no longer needs a bespoke dynamic-analysis and I/O-extraction stack per language pair; the per-language cost drops to the order of a few hundred lines of static analysis plus prompt text.
  • Translation bugs caused by inadequate test suites, such as byte-counting versus character-counting on Unicode strings, become detectable because the agent is instructed to generate tests that exercise the semantic differences the analyzers flagged.
  • Repair can be interleaved with validation: because a failing test and a semantic report are both available, the repair agent can iterate until previously failing project tests pass, recovering 50.6% of buggy translations instead of 18.5%.
  • Adding a new language is a matter of static-analysis glue plus prompts (about 280 lines of code per language pair), which makes the quadratic space of language pairs more reachable than with prior tools.
  • The multi-agent structure is compatible with different underlying LLMs and coding agents, so users can swap models without rebuilding the validation logic.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper deliberately does not audit equivalent (EQ) verdicts, so its overall accuracy claim is only partially grounded; a hidden false-equivalence rate is plausible and should be measured before relying on the system's 'equivalent' outputs in production.
  • The 60.7% correctness on disagreements comes from a sample audited by two authors who also built the competing tools; an independent, blinded audit with third-party reviewers would give a stronger estimate.
  • The same architecture could be pointed at same-language equivalence checking, regression testing, or migration safety checking, since the six semantic dimensions are not specific to cross-language translation.
  • Repair quality is judged only by previously failing project tests, with generated tests excluded to avoid bias, so the patch success rate depends on how well the original test suite covers each bug.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents MatchFixAgent, an LLM-based multi-agent framework for validating the functional equivalence of repository-level code translations and for repairing inequivalent translations. The framework combines six semantic analyses (control flow, data flow, I/O, library APIs, exceptions, specifications) with a test-generation and repair agent and a final verdict agent. The evaluation uses 2,219 translation pairs spanning six language pairs from four prior benchmark suites. The paper claims that MatchFixAgent produces a verdict for 99.2% of pairs, agrees with prior tools on 72.8% of pairs where both produce verdicts, is found correct in 60.7% of manually adjudicated disagreements, and repairs 50.6% of translation bugs versus 18.5% for prior techniques, at a development cost of roughly 1,650 lines of code.

Significance. If the validation-accuracy claims are supported, this would be a notable contribution to repository-level code translation: a language-agnostic approach with low engineering overhead, a credible multi-agent design, and empirical comparison over a substantially larger and more diverse benchmark than prior single-pair tools. The paper has concrete strengths: it assembles and reuses 2,219 translation pairs across six PL pairs; it ships an unusually lightweight implementation with documented per-language cost; it provides logs for external inspection; it includes an ablation study and an adaptability check with a different LLM and agent framework; and it reports repair results with a defined validation protocol using original failing tests. The main weakness is that the central accuracy claim is not measured for the majority of outputs: equivalent verdicts, which are 68.5% of all verdicts, are explicitly not checked against ground truth, and the disagreement-based 60.7% statistic cannot bound the false-EQ rate. The repair comparison also stacks conditions across tools, and the ablation uses a label that includes MatchFixAgent's own verdicts.

major comments (4)
  1. [Section 6 and Table 2] The paper explicitly states in Section 6 that 'we do not analyze the correctness of equivalent verdicts,' yet Table 2 shows that 1,519 of 2,219 (68.5%) of MatchFixAgent's verdicts are EQ. The abstract's claim of 'highly accurate validation results' is therefore unmeasured for the majority of outputs: the manual investigation covers only 145 disagreement cases, and the 60.7% disagreement-correct rate does not bound the false-EQ rate among agreed EQ verdicts. I ask for an oracle or substantial evidence on a sample of agreed EQ verdicts (for example, differential stress testing, mutation-based checks, or formal equivalence checks), or for claims to be revised to the level actually supported, namely accuracy on audited disagreements.
  2. [Section 4.2.2] The disagreement adjudication is not representative and is reported without statistical rigor. The paper filters out 11 RustRepoTrans disagreements that would have favored MatchFixAgent, leaving RustRepoTrans with only 12.5% of remaining disagreements ruled in favor of MatchFixAgent, and the overall 60.7% aggregates over the filtered set without re-weighting. The sample has 145 cases with no confidence interval, and the two author-adjudicators are co-authors of the competing tools (including Oxidizer and AlphaTrans), which creates a concrete risk of adjudication bias. Please report the unfiltered numbers, compute confidence intervals for the per-tool and overall disagreement-correct rates, and ideally add an independent adjudicator or a pre-registered ruling protocol.
  3. [Section 4.5.1 and Figures 7-8] The ablation study measures baseline 'validation accuracy' as the rate at which the standalone agent agrees with a label that includes MatchFixAgent's own verdicts on 1,091 non-dispute instances. Since the label is partly defined by MatchFixAgent itself, the reported 42.3% drop is at least in part a measure of self-consistency rather than correctness, and the same issue applies to the 'Accuracy (60.31%,100%)' comparison in Figure 8. This weakens the claim that the semantic analyzer and test generator improve validation accuracy. Please re-run the ablation against an external ground-truth sample, or re-frame the metric explicitly as 'agreement with the combined MatchFixAgent-plus-tool label' and temper the corresponding conclusions.
  4. [Section 4.3 and Table 3] The repair comparison is not apples-to-apples. The 18.5% baseline repair rate comes entirely from RustRepoTrans (49 repaired bugs), while Oxidizer, AlphaTrans, and Skel contribute zero tool repairs, and the paper does not report per-tool counts of the 265 pairs selected by the criterion 'where both techniques generated a patch.' Furthermore, patches are validated only against the original failing tests, which the paper itself argues are inadequate for equivalence detection. Please report the per-tool selection counts, clarify what 'Tool Repaired = 0' means for tools whose repair is interleaved with translation rather than exposed as a standalone phase, and validate a sample of MatchFixAgent patches with newly generated tests or manual semantic inspection beyond the 47 disagreement patches.
minor comments (6)
  1. [Section 3.2.1] The phrase 'to analyze the the control flow structures' contains a duplicated article and should be corrected.
  2. [Section 3.2.3] The sentence 'The LLM is prompted also prompted to produce' repeats 'prompted' and should be rewritten.
  3. [Section 4.2.2] The phrase 'MatchFixAgent's accuracy fairs worse' should be 'performs worse'.
  4. [Table 2] The text should state explicitly, at the first mention of the 72.8% agreement figure, that Agreement and Disagreement are computed only over pairs where both tools produce a verdict; this is currently left to the table caption.
  5. [Figure 7] The y-axis label 'Validation Accuracy' is misleading because the plotted quantity is agreement with a combined verdict label, not ground-truth accuracy; please rename the axis and clarify in the caption.
  6. [Section 6] The sentence 'While this means we don't have any measure of true accuracy of MatchFixAgent' should be reconciled with the abstract's 'highly accurate validation results' claim, since the current abstract overstates what the evaluation can support.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MatchFixAgent's validation and repair claims are evaluated against external benchmarks and independent test oracles; the paper's admitted gap on equivalent-verdict correctness is a validity limitation, not a circular derivation.

full rationale

The central validation comparison is not self-referential: MatchFixAgent is run on 2,219 source-translation pairs drawn from the artifacts of four prior techniques, and its verdicts are compared with those tools' verdicts and with original project tests. Repair patches are validated using original failing project tests, explicitly excluding MatchFixAgent-generated tests to avoid bias ('We did not use generated tests by MatchFixAgent'), so the repair result is not an artifact of the system's own outputs. The disagreement analysis uses manual adjudication (even though performed by the authors), and the paper candidly states in Section 6 that it does 'not analyze the correctness of equivalent verdicts' and therefore has no 'measure of true accuracy of MatchFixAgent.' That is an acknowledged validity gap in the headline accuracy claim, but it is not a circular step: no verdict, parameter, or metric is constructed from the quantity it is supposed to predict. The ablation study labels baseline 'accuracy' as agreement with the existing tool and MatchFixAgent on non-dispute instances; this is a self-comparison of the ablated agent to the full system, but it is presented as an ablation/fidelity measure rather than as an independent external validation, and it does not feed back into the paper's main validation or repair claims. Self-citations to AlphaTrans, Oxidizer, and VERT are used as baselines and benchmark sources, not as the sole justification of MatchFixAgent's correctness. Overall, no equation or constructed metric reduces to its own input, so under the stated rules the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The framework's results rest on a small set of empirically chosen thresholds and a strong delegation of semantic judgment to LLMs; no new mathematical entities are introduced.

free parameters (3)
  • CFG/DFG similarity threshold = 0.7
    Used to short-circuit the control flow and data flow analyzers without invoking the LLM; the value was chosen empirically, not derived from theory.
  • Timeout budget = 1000 seconds
    Set empirically after analyzing the execution time of 300 samples (Section 4.1.2).
  • Disagreement sampling quota = 5 per project per D1/D2
    For human adjudication, at most five disagreements were sampled per category per project (Section 4.2.2); the 60.7% correctness figure is an estimate from this sample.
assumptions (5)
  • domain assumption LLM-generated semantic analyses and test cases are reliable proxies for functional equivalence
    The entire framework delegates semantic comparison to LLMs (Section 3.2); if LLM analyses are systematically biased, verdicts and patches inherit the bias.
  • domain assumption The 1:1 functional equivalence definition matches developer intent for translated code
    Used throughout prompts (Figure 4); the paper acknowledges RustRepoTrans includes idiomatic refactors where strict 1:1 equivalence is disputed (Section 4.2.2).
  • domain assumption Original project tests are a valid oracle for validating repair patches
    Repair evaluation accepts a patch only when all originally failing tests pass (Section 4.3); tests may be incomplete or flaky.
  • domain assumption Tree-Sitter ASTs and the simple def-use data flow analysis are correct enough for the six languages
    CFG/DFG extraction underlies the semantic analyzers (Sections 3.2.1 and 3.2.2); the paper does not handle aliasing, concurrency, or context sensitivity.
  • standard math Jaccard similarity and edit distance are meaningful structural similarity measures for CFG/DFG equivalence
    Used to short-circuit analyzers (Section 3.2.1 and Algorithm 3); high similarity is assumed to imply semantic equivalence without formal proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MatchFixAgent: Language-Agnostic Autonomous Repository-Level Code Translation Validation and Repair." pith.science (2026). https://pith.science/paper/VLOPI4W7

@misc{pith2026250916187,
  author       = {Pith},
  title        = {Pith review of: MatchFixAgent: Language-Agnostic Autonomous Repository-Level Code Translation Validation and Repair},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VLOPI4W7}},
  note         = {Machine review of arXiv:2509.16187}
}
read the original abstract

Code translation transforms source code from one programming language (PL) to another. Validating the functional equivalence of translation and repairing, if necessary, are critical steps in code translation. Existing automated validation and repair approaches struggle to generalize to many PLs due to high engineering overhead, and they rely on existing and often inadequate test suites, which results in false claims of equivalence and ineffective translation repair. To bridge this gap, we develop MatchFixAgent, a large language model (LLM)-based, PL-agnostic framework for equivalence validation and repair of translations. MatchFixAgent features a multi-agent architecture that divides equivalence validation into several sub-tasks to ensure thorough and consistent semantic analysis of the translation. We compare MatchFixAgent's validation and repair results with four repository-level code translation techniques. Our results demonstrate that MatchFixAgent produces (in)equivalence verdicts for 99.2% of translation pairs, with the same equivalence validation result as prior work on 72.8% of them. When MatchFixAgent's result disagrees with prior work, we find that 60.7% of the time MatchFixAgent's result is actually correct. In addition, we show that MatchFixAgent can repair 50.6% of inequivalent translation, compared to prior work's 18.5%.

Figures

Figures reproduced from arXiv: 2509.16187 by the authors.

Figure 1
Figure 1. Illustration of key limitations of existing techniques in validation and repair of repository-level code [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Overview of MatchFixAgent. it automatically generates a patch that uses .chars().count() to ensure proper handling of both ASCII and Unicode characters ( b3 ). Limitation 3: Repair. Current translation repair techniques solely rely on simple feedback￾driven approaches, which have proven inadequate in practical settings. Skel [73], Oxidizer [86], and Syzygy [53] utilize multi-turn iterative prompting techniques. Alph… view at source ↗
Figure 3
Figure 3. CFG and DFP structures extracted by the Semantic Analyzer component in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Prompt structure of Test Generator and Repair Agent. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Development cost of MatchFixAgent compared against existing tools. 4.4.1 Development Cost [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 7
Figure 7. Figure 7: Impact of semantic analyzer and test generator agent in [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 6
Figure 6. Figure 6: Agreement and dispute cases be￾tween tool validation system and Match￾FixAgent with Claude and Codex agents [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 8
Figure 8. Figure 8: Removing the semantic analyzer de￾creases the effectiveness of MatchFixAgent, while increasing token consumption, num￾ber of turns, and processing time. Translation Validation and Repair. Existing auto￾mated translation validation techniques either rely on test executi…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Mostly Automatic Translation of Language Interpreters from C to Safe Rust

    cs.PL 2026-06 unverdicted novelty 6.0 of 10

    Reboot translates six C interpreters (6k-23k LOC) to safe Rust with 1-11 interventions each, passing all original tests and 62-92% on new validation tests while removing memory vulnerabilities.

Reference graph

Works this paper leans on

90 extracted references · 48 canonical work pages · cited by 1 Pith paper

  1. [1]

    Muhammad Salman Abid, Mrigank Pawagi, Sugam Adhikari, Xuyan Cheng, Ryed Badr, Md Wahiduzzaman, Vedant Rathi, Ronghui Qi, Choiyin Li, Lu Liu, Rohit Sai Naidu, Licheng Lin, Que Liu, Asif Zubayer Palak, Mehzabin Haque, Xinyu Chen, Darko Marinov, and Saikat Dutta. 2024. GlueTest: Testing Code Translation via Language Interoperability. In2024 IEEE Internationa...

    work page arXiv 2024

  2. [2]

    Aider AI. 2025. AI pair programming in your terminal. https://aider.chat/

    2025

  3. [3]

    The Algorithms. 2025. All Algorithms implemented in Python. https://github.com/TheAlgorithms/Python/blob/ master/data_structures/binary_tree/binary_search_tree_recursive.py

    2025

  4. [4]

    The Algorithms. 2025. All Algorithms implemented in Python. https://github.com/TheAlgorithms/Python/blob/ master/data_structures/binary_tree/red_black_tree.py

    2025

  5. [5]

    Amazon. 2025. Amazon Q Developer. https://aws.amazon.com/q/developer/

    2025

  6. [6]

    Anthropic. 2025. Building Effective AI Agents. https://www.anthropic.com/engineering/building-effective-agents

    2025

  7. [7]

    David Belicza. 2025. TextRank on Go. https://github.com/DavidBelicza/TextRank

    2025

  8. [8]

    The SWE bench Team. 2025. SWE-bench Leaderboard. https://www.swebench.com/

    2025

Show all 90 references
  1. [9]

    Hugo Bollon. 2025. Go-edlib : Edit distance and string comparison library. https://github.com/hbollon/go-edlib

    2025

  2. [10]

    Islem Bouzenia, Premkumar Devanbu, and Michael Pradel. 2024. Repairagent: An autonomous, llm-based agent for program repair.arXiv preprint arXiv:2403.17134(2024)

    2024 arXiv

  3. [11]

    Xuemeng Cai, Jiakun Liu, Xiping Huang, Yijun Yu, Haitao Wu, Chunmiao Li, Bo Wang, Imam Nur Bani Yusuf, and Lingxiao Jiang. 2025. RustMap: Towards Project-Scale C-to-Rust Migration via Program Analysis and LLM.arXiv preprint arXiv:2503.17741(2025)

    2025 arXiv

  4. [12]

    Sung-Hyuk Cha. [n. d.]. Comprehensive survey on distance/similarity measures between probability density functions. City1, 2 ([n. d.]), 1. MatchFixAgent: Language-Agnostic Autonomous Repository-Level Code Translation Validation and Repair 19

  5. [13]

    Delta Chat. 2025. Delta.Chat C-Library with e2e chat-over-email functionality & Python bindings. https://github. com/deltachat/deltachat-core

    2025

  6. [14]

    Jimenez, John Yang, Leyton Ho, Tejal Patwardhan, Kevin Liu, and Aleksander Madry

    Neil Chowdhury, James Aung, Chan Jun Shern, Oliver Jaffe, Dane Sherburn, Giulio Starace, Evan Mays, Rachel Dias, Marwan Aljubeh, Mia Glaese, Carlos E. Jimenez, John Yang, Leyton Ho, Tejal Patwardhan, Kevin Liu, and Aleksander Madry. 2024. Introducing SWE-bench Verified. https:...

    2024

  7. [15]

    Cognition. 2025. Introducing Devin, the first AI software engineer. https://cognition.ai/blog/introducing-devin

    2025

  8. [16]

    Vivid Cortex. 2025. gohistogram - Histograms in Go. https://github.com/VividCortex/gohistogram

    2025

  9. [17]

    Hasan Ferit Eniser, Hanliang Zhang, Cristina David, Meng Wang, Maria Christakis, Brandon Paulsen, Joey Dodds, and Daniel Kroening. 2024. Towards translating real-world code with llms: A study of translating to rust.arXiv preprint arXiv:2405.11514(2024)

    2024 arXiv

  10. [18]

    Montana Flynn. 2025. Stats - Golang Statistics Package. https://github.com/montanaflynn/stats

    2025

  11. [19]

    The Apache Software Foundation. 2025. AMCL - Apache Milagro Crypto Library. https://github.com/apache/incubator- milagro-crypto-c

    2025

  12. [20]

    The Apache Software Foundation. 2025. Apache Commons CLI. https://github.com/apache/commons-cli

    2025

  13. [21]

    The Apache Software Foundation. 2025. Apache Commons CSV. https://github.com/apache/commons-csv

    2025

  14. [22]

    The Apache Software Foundation. 2025. Apache Commons FileUpload. https://github.com/apache/commons-fileupload

    2025

  15. [23]

    The Apache Software Foundation. 2025. Apache Commons Validator. https://github.com/apache/commons-validator

    2025

  16. [24]

    The Apache Software Foundation. 2025. Apache Iceberg. https://github.com/apache/iceberg

    2025

  17. [25]

    The Apache Software Foundation. 2025. Apache PyIceberg. https://github.com/apache/iceberg-python

    2025

  18. [26]

    The Apache Software Foundation. 2025. MCJL - Milagro Crypto Java Library. https://github.com/apache/incubator- milagro-java

    2025

  19. [27]

    Patrice Godefroid, Michael Y Levin, David A Molnar, et al. 2008. Automated whitebox fuzz testing.. InNdss, Vol. 8. 151–166

    2008

  20. [28]

    André Hora, Romain Robbes, Nicolas Anquetil, Anne Etien, Stéphane Ducasse, and Marco Tulio Valente. 2015. How do developers react to API evolution? The Pharo ecosystem case. In2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). 251–260. doi:10.110...

    2015

  21. [29]

    Ali Reza Ibrahimzada, Kaiyao Ke, Mrigank Pawagi, Muhammad Salman Abid, Rangeet Pan, Saurabh Sinha, and Reyhaneh Jabbarvand. 2025. AlphaTrans: A Neuro-Symbolic Compositional Approach for Repository-Level Code Translation and Validation.Proc. ACM Softw. Eng.2, FSE, Article FSE10...

    2025 doi

  22. [30]

    Suman Jain and Inderveer Chana. 2015. Modernization of Legacy Systems: A Generalised Roadmap. InProceedings of the Sixth International Conference on Computer and Communication Technology 2015(Allahabad, India)(ICCCT ’15). Association for Computing Machinery, New York, NY, USA,...

    2015

  23. [31]

    Pooyan Jamshidi, Aakash Ahmad, and Claus Pahl. 2013. Cloud Migration Research: A Systematic Review.IEEE Transactions on Cloud Computing1, 2 (2013), 142–157. doi:10.1109/TCC.2013.10

    2013 doi

  24. [32]

    Jawah. 2025. Charset Normalizer: Truly universal encoding detector in pure Python. https://github.com/jawah/ charset_normalizer

    2025

  25. [33]

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench: Can Language Models Resolve Real-world Github Issues?. InThe Twelfth International Conference on Learning Representations. https://openreview.net/forum?...

    2024

  26. [34]

    Batlajery, Amir M

    Ravi Khadka, Belfrit V. Batlajery, Amir M. Saeidi, Slinger Jansen, and Jurriaan Hage. 2014. How do professionals perceive legacy systems and software modernization?. InProceedings of the 36th International Conference on Software Engineering(Hyderabad, India)(ICSE 2014). Associ...

    2014

  27. [35]

    Musawwer Khan, Islam Ali, Wasif Nisar, Muhammad Qaiser Saleem, Ali S Ahmed, Haysam E Elamin, Waqar Mehmood, and Muhammad Shafiq. 2022. Modernization Framework to Enhance the Security of Legacy Information Systems. Intelligent Automation & Soft Computing32, 1 (2022), 543–555. d...

    2022

  28. [36]

    George Klees, Andrew Ruef, Benji Cooper, Shiyi Wei, and Michael Hicks. 2018. Evaluating fuzz testing. InProceedings of the 2018 ACM SIGSAC conference on computer and communications security. 2123–2138

    2018

  29. [37]

    German, Ali Ouni, Takashi Ishio, and Katsuro Inoue

    Raula Gaikovina Kula, Daniel M. German, Ali Ouni, Takashi Ishio, and Katsuro Inoue. 2018. Do developers update their library dependencies?Empirical Softw. Engg.23, 1 (Feb. 2018), 384–417. doi:10.1007/s10664-017-9521-5

    2018 doi

  30. [38]

    libp2p. 2025. The Python implementation of the libp2p networking stack. https://github.com/libp2p/py-libp2p

    2025

  31. [39]

    Junwei Liu, Kaixin Wang, Yixuan Chen, Xin Peng, Zhenpeng Chen, Lingming Zhang, and Yiling Lou. 2024. Large language model-based agents for software engineering: A survey.arXiv preprint arXiv:2409.02977(2024)

    2024 arXiv

  32. [40]

    ZhouYang Luo. 2025. A library implementing different string similarity and distance measures using Python. https: //github.com/luozhouyang/python-string-similarity/tree/master/strsimpy

    2025

  33. [41]

    Nickil Maveli, Antonio Vergari, and Shay B Cohen. 2025. What can Large Language Models Capture about Code Functional Equivalence?. InFindings of the Association for Computational Linguistics: NAACL 2025, Luis Chiruzzo, 20 Ali Reza Ibrahimzada, Brandon Paulsen, Reyhaneh Jabbarv...

    2025 doi

  34. [42]

    Miller, Agnes F

    Frederic P. Miller, Agnes F. Vandome, and John McBrewster. 2009.Levenshtein Distance: Information theory, Computer science, String (computer science), String metric, Damerau?Levenshtein distance, Spell checker, Hamming distance. Alpha Press

    2009

  35. [43]

    Vikram Nitin, Rahul Krishna, and Baishakhi Ray. 2024. Spectra: Enhancing the code translation ability of language models by generating multi-modal specifications.arXiv preprint arXiv:2405.18574(2024)

    2024

  36. [44]

    Vikram Nitin, Rahul Krishna, Luiz Lemos do Valle, and Baishakhi Ray. 2025. C2SaferRust: Transforming C Projects into Safer Rust with NeuroSymbolic Techniques.arXiv preprint arXiv:2501.14257(2025)

    2025

  37. [45]

    OpenAI. 2025. Introducing OpenAI o3 and o4-mini. https://openai.com/index/introducing-o3-and-o4-mini/

    2025

  38. [46]

    Oracle. 2025. GraalVM. https://www.graalvm.org

    2025

  39. [47]

    Guangsheng Ou, Mingwei Liu, Yuxuan Chen, Xin Peng, and Zibin Zheng. 2024. Repository-level Code Translation Benchmark Targeting Rust.arXiv preprint arXiv:2411.13990(2024)

    2024

  40. [48]

    Siru Ouyang, Wenhao Yu, Kaixin Ma, Zilin Xiao, Zhihan Zhang, Mengzhao Jia, Jiawei Han, Hongming Zhang, and Dong Yu. 2025. RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph. InThe Thirteenth International Conference on Learning Representations. https...

    2025

  41. [49]

    Rangeet Pan, Ali Reza Ibrahimzada, Rahul Krishna, Divya Sankar, Lambert Pouguem Wassi, Michele Merler, Boris Sobolev, Raju Pavuluri, Saurabh Sinha, and Reyhaneh Jabbarvand. 2024. Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code. I...

    2024

  42. [50]

    Will Pearson. 2025. Python lib for TOML. https://github.com/uiri/toml/tree/master/toml

    2025

  43. [51]

    James Polera. 2025. gonameparts. https://github.com/polera/gonameparts

    2025

  44. [52]

    Haifeng Ruan, Yuntong Zhang, and Abhik Roychoudhury. 2025. SpecRover: Code Intent Extraction via LLMs. In2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). 963–974. doi:10.1109/ICSE55347.2025.00080

    2025

  45. [53]

    Manish Shetty, Naman Jain, Adwait Godbole, Sanjit A Seshia, and Koushik Sen. 2024. Syzygy: Dual Code-Test C to (safe) Rust Translation using LLMs and Dynamic Analysis.arXiv preprint arXiv:2412.14234(2024)

    2024 arXiv

  46. [54]

    The Anthropic Team. 2025. Claude. https://www.anthropic.com/claude

    2025

  47. [55]

    The BigCodeBench Team. 2025. BigCodeBench Leaderboard. https://bigcode-bench.github.io/

    2025

  48. [56]

    The Claude Code Team. 2025. Claude Code. https://github.com/anthropics/claude-code

    2025

  49. [57]

    The GNU Team. 2025. C Compiler. https://gcc.gnu.org/

    2025

  50. [58]

    The Google Deepmind Team. 2025. Gemini Pro. https://deepmind.google/models/gemini/pro/

    2025

  51. [59]

    The Go Language Team. 2025. Go Language. https://go.dev/

    2025

  52. [60]

    The Java Language Team. 2025. Java Language. https://www.java.com/en/

    2025

  53. [61]

    The Moatless Tools Team. 2025. Moatless Tools. https://github.com/aorwall/moatless-tools

    2025

  54. [62]

    The NodeJS Team. 2025. NodeJS. https://nodejs.org/en

    2025

  55. [63]

    The OpenAI Team. 2025. GPT-4o. https://openai.com/index/hello-gpt-4o/

    2025

  56. [64]

    The OpenAI Team. 2025. OpenAI Codex CLI. https://github.com/openai/codex

    2025

  57. [65]

    The Python Team. 2025. Conversion functions between RGB and other color systems. https://github.com/python/ cpython/blob/3.13/Lib/colorsys.py

    2025

  58. [66]

    The Python Team. 2025. CPython. https://github.com/python/cpython

    2025

  59. [67]

    The Python Team. 2025. Heap queue algorithm (a.k.a. priority queue). https://github.com/python/cpython/blob/3.13/ Lib/heapq.py

    2025

  60. [68]

    The Python Team. 2025. A parser for HTML and XHTML. https://github.com/python/cpython/blob/3.13/Lib/html/ parser.py

    2025

  61. [69]

    The Python Language Team. 2025. Python Language. https://www.python.org/

    2025

  62. [70]

    The Rust Language Team. 2025. Rust Language. https://www.rust-lang.org/

    2025

  63. [71]

    Osamu Tonomori. 2025. Checkdigit. https://github.com/osamingo/checkdigit

    2025

  64. [72]

    Tree-Sitter. 2025. Tree-Sitter Library. https://tree-sitter.github.io/tree-sitter/

    2025

  65. [73]

    Bo Wang, Tianyu Li, Ruishi Li, Umang Mathur, and Prateek Saxena. 2025. Program Skeletons for Automated Program Translation.Proc. ACM Program. Lang.9, PLDI, Article 184 (June 2025), 25 pages. doi:10.1145/3729287

    2025 doi

  66. [74]

    Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brenna...

    2025

  67. [75]

    Ying Wang, Bihuan Chen, Kaifeng Huang, Bowen Shi, Congying Xu, Xin Peng, Yijian Wu, and Yang Liu. 2020. An Empirical Study of Usages, Updates and Risks of Third-Party Libraries in Java Projects. In2020 IEEE International Conference on Software Maintenance and Evolution (ICSME)...

    2020

  68. [76]

    Anjiang Wei, Jiannan Cao, Ran Li, Hongyu Chen, Yuhui Zhang, Ziheng Wang, Yuan Liu, Thiago SFX Teixeira, Diyi Yang, Ke Wang, et al. 2025. EquiBench: Benchmarking Large Language Models’ Understanding of Program Semantics via Equivalence Checking.arXiv preprint arXiv:2502.12466(2025)

    2025

  69. [77]

    Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I Wang. 2025. SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution.arXiv preprint arXiv:2502.18449(2025)

    2025 arXiv

  70. [78]

    Luke Weiler. 2025. Basic Math. https://github.com/lukew3/mathgenerator/blob/main/mathgenerator/basic_math.py

    2025

  71. [79]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, W...

    2025

  72. [80]

    Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2025. Demystifying LLM-Based Software Engineering Agents.Proc. ACM Softw. Eng.2, FSE, Article FSE037 (June 2025), 24 pages. doi:10.1145/3715754

    2025 doi

  73. [81]

    Pengyu Xue, Linhao Wu, Zhen Yang, Chengyi Wang, Xiang Li, Yuxiang Zhang, Jia Li, Ruikai Jin, Yifei Pei, Zhaoyan Shen, Xiran Lyu, and Jacky Wai Keung. 2025. ClassEval-T: Evaluating Large Language Models in Class-Level Code Translation.Proc. ACM Softw. Eng.2, ISSTA, Article ISST...

    2025 doi

  74. [82]

    Aidan ZH Yang, Yoshiki Takashima, Brandon Paulsen, Josiah Dodds, and Daniel Kroening. 2024. VERT: Verified Equivalent Rust Transpilation with Large Language Models as Few-shot Learners.arXiv preprint arXiv:2404.18852 (2024)

    2024 arXiv

  75. [83]

    John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik R Narasimhan, and Ofir Press

  76. [84]

    John Yang, Carlos E Jimenez, Alex L Zhang, Kilian Lieret, Joyce Yang, Xindi Wu, Ori Press, Niklas Muennighoff, Gabriel Synnaeve, Karthik R Narasimhan, Diyi Yang, Sida Wang, and Ofir Press. 2025. SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?. InThe ...

    2025

  77. [85]

    Zhen Yang, Fang Liu, Zhongxing Yu, Jacky Wai Keung, Jia Li, Shuo Liu, Yifan Hong, Xiaoxue Ma, Zhi Jin, and Ge Li

  78. [86]

    Hanliang Zhang, Cristina David, Meng Wang, Brandon Paulsen, and Daniel Kroening. 2025. Scalable, Validated Code Translation of Entire Projects using Large Language Models.Proc. ACM Program. Lang.9, PLDI, Article 212 (June 2025), 26 pages. doi:10.1145/3729315

    2025 doi

  79. [87]

    ACM Softw

    Exploring and Unleashing the Power of Large Language Models in Automated Code Translation.Proc. ACM Softw. Eng.1, FSE, Article 71 (July 2024), 24 pages. doi:10.1145/3660778

    2024 doi

  80. [88]

    Celal Ziftci, Stoyan Nikolov, Anna Sjövall, Bo Kim, Daniele Codecasa, and Max Kim. 2025. Migrating Code At Scale With LLMs At Google. InProceedings of the 33rd ACM International Conference on the Foundations of Software Engineering (Clarion Hotel Trondheim, Trondheim, Norway)(...

    2025

  81. [89]

    Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. AutoCodeRover: Autonomous Program Improvement. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (Vienna, Austria)(ISSTA 2024). Association for Computing Machiner...

    2024

  82. [2024]

    InThe Thirty-eighth Annual Conference on Neural Information Processing Systems

    SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems. https://openreview.net/forum?id=mXpq6ut8J3

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.