REVIEW 6 cited by
Towards Translating Real-World Code with LLMs: A Study of Translating to Rust
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) show promise in code translation - the task of translating code written in one programming language to another language - due to their ability to write code in most programming languages. However, LLM's effectiveness on translating real-world code remains largely unstudied. In this work, we perform the first substantial study on LLM-based translation to Rust by assessing the ability of five state-of-the-art LLMs, GPT4, Claude 3, Claude 2.1, Gemini Pro, and Mixtral. We conduct our study on code extracted from real-world open source projects. To enable our study, we develop FLOURINE, an end-to-end code translation tool that uses differential fuzzing to check if a Rust translation is I/O equivalent to the original source program, eliminating the need for pre-existing test cases. As part of our investigation, we assess both the LLM's ability to produce an initially successful translation, as well as their capacity to fix a previously generated buggy one. If the original and the translated programs are not I/O equivalent, we apply a set of automated feedback strategies, including feedback to the LLM with counterexamples. Our results show that the most successful LLM can translate 47% of our benchmarks, and also provides insights into next steps for improvements.
Forward citations
Cited by 6 Pith papers
-
Towards Reliable C-to-Rust Translation with Rule-Guided Reasoning and Reinforcement Learning
A rule-guided MCTS plus dual-reward reinforcement learning pipeline improves LLM-based C-to-Rust translation accuracy and cuts unsafe Rust output across three benchmarks.
-
His2Trans: A Knowledge-Guided Agentic Framework for Project-Level C-to-Rust Migration
Combining build-trace-derived skeletons with a knowledge base of historical translation pairs raises C-to-Rust compilation pass rates and lowers unsafe-code ratios, but on general benchmarks functional-correctness sco...
-
Vibration-Based Energy Metric for Restoring Needle Alignment in Autonomous Robotic Ultrasound
A vibration-based energy metric, obtained by periodically vibrating the needle, is used as an out-of-plane feedback signal to re-align the ultrasound probe with the needle plane, reporting 0.41 mm translation and 0.51...
-
Can LLMs Replace Humans During Code Chunking?
LLM-generated partitions of legacy code yield documentation that LLM judges rate as up to 20% more factual and up to 10% more useful than documentation based on human expert partitions.
-
CODEMENV: Benchmarking Large Language Models on Code Migration
CODEMENV provides 922 examples and three tasks for evaluating LLMs on cross-version code migration, finding models are much better at migrating old code to new versions (up to 43.84% pass@1) than the reverse.
-
From C to Idiomatic Rust: A Ship-of-Theseus Agentic Translation
A C-to-Rust migration methodology that separates lossless c2rust transpilation from agentic per-function rewriting with compilation and behavioral gates is demonstrated on the 12.5 kSLOC iodine DNS tunnel.
Discussion (0). Continue with ORCID to comment.