REVIEW 6 cited by
LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent research explores optimization using large language models (LLMs) by either iteratively seeking next-step solutions from LLMs or directly prompting LLMs for an optimizer. However, these approaches exhibit inherent limitations, including low operational efficiency, high sensitivity to prompt design, and a lack of domain-specific knowledge. We introduce LLaMoCo, the first instruction-tuning framework designed to adapt LLMs for solving optimization problems in a code-to-code manner. Specifically, we establish a comprehensive instruction set containing well-described problem prompts and effective optimization codes. We then develop a novel two-phase learning strategy that incorporates a contrastive learning-based warm-up procedure before the instruction-tuning phase to enhance the convergence behavior during model fine-tuning. The experiment results demonstrate that a CodeGen (350M) model fine-tuned by our LLaMoCo achieves superior optimization performance compared to GPT-4 Turbo and the other competitors across both synthetic and realistic problem sets. The fine-tuned model and the usage instructions are available at https://anonymous.4open.science/r/LLaMoCo-722A.
Forward citations
Cited by 6 Pith papers
-
Search Hardness-Aware LLM-Based Problem Formulation for Expensive Simulation-Driven Design
SHA-PF uses initial simulation data to select a 'hard but promising' anchor satisfaction state and evolves LLM-generated formulations that prioritize it, reaching target designs with fewer expensive simulations on ant...
-
Efficient Heuristics Generation for Solving Combinatorial Optimization Problems Using Large Language Models
Hercules improves LLM-based heuristic generation by abstracting core components of elite heuristics into prompts, and Hercules-P reduces search cost by having the LLM predict heuristic fitness values instead of evalua...
-
ConfigX: Modular Configuration for Evolutionary Algorithms via Multitask Reinforcement Learning
A unified RL policy can configure modular evolutionary algorithms within a family, but the claimed universal zero-shot generalization across algorithm families is not supported.
-
ParaStudent: Generating and Evaluating Realistic Student Code by Teaching LLMs to Struggle
Fine-tuning a 7B code model on real student submissions produces code that better matches student error patterns, style, and incremental revision trajectories than prompting-only models across two temporal resolutions.
-
Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings
A string-based, metadata-guided language model framework for universal offline black-box optimization, with two variants and two embedding regularizations.
-
A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving
A literature survey that classifies LLM-based optimization research into modeling and solving, with solving divided into LLMs as optimizers, low-level components, and high-level managers.
Discussion (0). Continue with ORCID to comment.