e-SimFT: Alignment of Generative Models with Simulation Feedback for Pareto-Front Design Exploration

Amir Hosein Khasahmadi; Hyunmin Cheong; Mohammadmehdi Ataei; Pradeep Kumar Jayaraman

REVIEW 4 major objections 5 minor 12 references

e-SimFT: Alignment of Generative Models with Simulation Feedback for Pareto-Front Design Exploration

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that fine-tuning a generative design model with simulator feedback, then sampling it with epsilon-perturbed requirement values, produces Pareto fronts that beat recent multi-objective alignment baselines on hypervolume.

desk verdict A genuinely new adaptation of LLM preference alignment to generative engineering design with simulator feedback, but the central superiority claim is under-supported by the statistics as reported. read the letter →

arxiv 2502.02628 v1 pith:R2KFUJLP submitted 2025-02-04 cs.LG cs.AI

Hyunmin Cheong , Mohammadmehdi Ataei , Amir Hosein Khasahmadi , Pradeep Kumar Jayaraman This is my paper · ORCID

classification cs.LGcs.AI

keywords generativedesignParetofrontpreferencealignmentsimulationfeedbackdirectoptimizationepsilon-constraintmethodhypervolumegeartrain

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

e-SimFT is a framework for making a conditional generative design model respond to a specific design requirement and for sampling the fine-tuned models to trace out a Pareto front of best-compromise solutions. It replaces the human preference labels used in large-language-model alignment with feedback from a physics simulator, which can cheaply and accurately evaluate any proposed design. The paper demonstrates the recipe on GearFormer, a transformer-based gear-train designer, using two original requirements (speed ratio, output motion position) and two new ones (cost, bounding box volume). In experiments with 30 test problems at sampling budgets of 30 and 300, e-SimFT produces Pareto fronts with higher average hypervolume than the Rewarded Soup and Rewards-in-Context baselines in both two- and three-requirement scenarios. The takeaway is that simulator-grounded alignment, not human annotation, can be enough to steer generative models toward user-prioritized trade-offs.

What carries the argument

The mechanism has two halves. SimFT is the alignment half: a simulator evaluates designs to produce filtered supervised datasets, preference pairs, or per-sample rewards, and the generative model is fine-tuned with SFT, DPO, or PPO. Epsilon-sampling is the exploration half: given a requirement $r_i$, the framework samples from the model fine-tuned for $r_i$ while incrementing the target value by $\epsilon \in [-5,5]$ for original requirements or $[0,10]$ for new ones, mimicking the epsilon-constraint method for classical multi-objective optimization. The load-bearing idea, stated in Section 4.3, is that a model fine-tuned for $r_i$ can best enforce that constraint, so sampling it with perturbed values is equivalent to posing $r_i$ as a constraint and treating the rest as objectives. Hypervolume, computed from normalized constraint violations, is the quality metric that ranks the methods.

What would settle it

Run e-SimFT and the baselines on a test suite where the true Pareto front is known from exhaustive enumeration or a classical multi-objective optimizer, then measure the constraint-violation rate of the epsilon-sampled designs against their original target values. If the perturbed samples systematically violate the original constraints, or if the hypervolume relative to the true front is no better than the baselines, the framework's core benefit disappears.

Watch

Extended reading notes

Core claim

The central claim is that preference-alignment methods developed for LLMs can be repurposed for engineering design when the reward or preference signal comes from a simulator. For original requirements (those the generative model was trained on), supervised fine-tuning on simulation-filtered data is presented as sufficient, because anything less than a perfect match would harm the pretrained model. For new requirements the model has never seen, the paper's recipe is SFT to train a new encoder plus DPO or PPO with simulator rewards, with DPO performing slightly better. The second half of the claim is that a sampling scheme inspired by the epsilon-constraint method—varying the target requirement value by small $\epsilon$ steps and sampling the corresponding SimFT model—turns the fine-tuned models into a high-quality Pareto front. Across the tables, e-SimFT has the highest mean hypervolume in every aggregated two- and three-requirement comparison, and its advantage persists when the per-scenario sample budget is raised from 30 to 300.

Load-bearing premise

The method assumes that when you nudge the requirement value by a small amount, the fine-tuned model still produces designs that truly respect the original constraint, so the perturbed samples land on the real Pareto front instead of drifting off it.

Editorial extensions

If this is right

Engineers can get a Pareto front from a generative design model using only a simulator and a modest fine-tuning dataset, without retraining from scratch.
New requirements never seen by the original model (such as cost or bounding box volume) can be imposed by fine-tuning the decoder and adding a new encoder, followed by DPO or PPO.
The hypervolume advantage of e-SimFT persists when the sampling budget grows tenfold, so the improvement is not merely a small-sample artifact.
The ablation shows that simulation-filtered data is essential for original requirements: accepting every generated sample drops speed and position accuracy below the pretrained baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the epsilon-sampling trick only needs a queryable simulator and a conditional generative model, it should transfer to other engineering domains and to scientific surrogates where outcomes are computable, not just gear-train design.
The DPO/PPO checkpoints were selected post hoc to keep valid designs above 95%; a production version would need an automatic validation criterion to avoid picking checkpoints that overfit the preference loss.
The heuristic that each SimFT model is the best enforcer of its own requirement could be made robust by learning a per-requirement constraint-violation predictor and using it to decide which model to sample for each epsilon level.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

An honest first attempt at something new: adapting LLM preference alignment to generative design with simulator feedback, and wrapping it in an epsilon-sampling scheme. The first application claim checks out against the cited literature, and the method presentation is clear and reproducible in spirit. The strongest parts are the ablations—showing that unfiltered SFT degrades original-requirement performance, and that DPO over-training trades validity for requirement satisfaction. Those results tell you something real about where SimFT belongs in the toolbox.

The soft spot is the headline comparison. The conclusion states that e-SimFT outperformed Rewarded Soup and Rewards-in-Context in both two- and three-requirement scenarios, but the Tables 2 and 3 means are small relative to the spread and no significance testing is reported. For instance, over 30 test problems the two-requirement mean advantage over RiC is roughly 0.04 with a standard deviation of about 0.22. e-SimFT is also not the best method in several individual scenarios (position:b.box, cost:b.box at N=30). That by itself doesn't kill the paper, but it means the central empirical claim isn't established. The authors should run paired tests or bootstrap confidence intervals and report per-problem wins/losses. The appendix also says the DPO/PPO checkpoints were picked post hoc by 'best requirement improvement' subject to validity, evaluated on the test problems used in the final comparison. That is selection on the test set unless the validation set was used for checkpointing; the paper should clarify and, better, use a held-out validation split for that decision. Finally, the evaluation is single-domain and the whole stack (base model, simulator, data) comes from the authors' earlier GearFormer work. That restricts generality, though the comparison against external multi-objective alignment baselines is fair.

On the stress-test note: I don't agree that epsilon-sampling is a weak point in itself. The epsilon-constraint inspiration is sensible, and the ablations show it adds value beyond SimFT or epsilon-sampling alone. The weak point is purely the statistics around the headline result.

Verdict: worth sending to peer review. The concept is substantive, the analysis gaps are fixable, and a good referee can push the authors to either establish the superiority claim properly or soften it. I'd cite this as an early application of simulator-based alignment in design exploration once the comparison is cleaned up.

Referee Report

4 major / 5 minor

Summary. The paper introduces e-SimFT, a framework for fine-tuning a pre-trained generative model for engineering design (GearFormer) using simulation feedback, and then sampling the fine-tuned models with a proposed method called epsilon-sampling to construct Pareto fronts over multiple design requirements. The authors adapt LLM alignment techniques (SFT, DPO, PPO) to engineering design, distinguishing original equality-constraint requirements from new inequality-constraint requirements. They evaluate the resulting SimFT models on individual requirement satisfaction and evaluate e-SimFT against two multi-objective alignment baselines (Rewarded Soup and Rewards-in-Context) plus ablations, reporting hypervolume means over 30 test problems and 10 trade-off scenarios. The paper claims that e-SimFT outperforms the baselines in both two- and three-requirement scenarios.

Significance. If the empirical claims are substantiated, the paper would make a useful contribution by demonstrating that preference-alignment techniques developed for LLMs can be transferred to generative models in engineering design, using a simulator as an accurate and scalable feedback source. The proposed epsilon-sampling heuristic is a plausible bridge between classical multi-objective optimization and generative-model sampling. The paper also provides a fairly detailed ablation study (rejection sampling for SFT, DPO vs PPO, checkpoint selection), and it reports means and standard deviations over 30 problems, which is a step toward reproducibility. However, the central quantitative claim of 'outperforming' the baselines is not statistically established, and the manuscript contains internal inconsistencies in its own scenario-by-scenario reporting. The strengths are the clear methodological framework and the concrete evaluation on a nontrivial design problem; the weakness is that the evidence as presented does not yet support the strength of the conclusions drawn.

major comments (4)

[§5.3, Tables 2 and 3; §6 Conclusions] The central claim that e-SimFT 'outperformed latest multi-objective alignment methods' is not supported by the reported statistics. The differences in mean hypervolume are small relative to the reported standard deviations (e.g., Table 2 two-requirement mean: e-SimFT 0.621±0.223 vs. RiC 0.580±0.227; three-requirement mean: 0.473±0.219 vs. 0.427±0.224). No paired significance tests are reported, and no per-problem results are provided, so the reader cannot assess whether the observed average gains are consistent across the 30 test problems. I request paired tests or per-problem data (e.g., violin plots or a table of individual hypervolumes) and a more measured wording of the conclusion.
[§5.3, Table 2, cost:b.box and position:b.box rows] The text says 'Only in one scenario an alternative multi-objective alignment method, RiC, achieved the highest hypervolume.' This statement is contradicted by Table 2: in the cost:b.box scenario e-SimFT (0.521) is below both RiC (0.580) and the baseline (0.571), and in the position:b.box scenario e-SimFT (0.537) is below RiC (0.555). The same is true in Table 3 (cost:b.box: e-SimFT 0.666 vs. RiC 0.703). The internal inconsistency between the claimed 'overall superior' performance and the scenario-level numbers needs to be resolved by either correcting the claim or providing an explanation for why these specific scenarios are exceptions.
[§4.3, epsilon-sampling; §5.1 experimental setup] The epsilon-sampling heuristic rests on the assumption that 'a model fine-tuned for ri can best enforce that constraint; therefore, sampling from that model would be equivalent to posing ri as a constraint and the rest of requirements as objectives.' This is a heuristic with no formal justification, and the paper does not verify empirically whether the samples obtained by perturbing the requirement value actually lie on or near the true Pareto front. In addition, the epsilon range for original requirements ([-5,5]) and for new requirements ([0,10]) are free hyperparameters that appear to be chosen without sensitivity analysis. I recommend adding a diagnostic that plots the generated fronts relative to known Pareto fronts (or at least a quantitative measure of Pareto-optimality, such as the IGD), and a brief sensitivity study over the epsilon range.
[§A.2, DPO/PPO training details] The checkpoint selection procedure for DPO and PPO ('the model checkpoint with the best requirement improvement was picked post hoc, subject to the criterion that 95% of the generated solutions are valid') is not a standard validation-based criterion and could lead to overfitting to the test scenario. The manuscript should clarify whether the requirement improvement is computed on a held-out validation set (as done for SFT) or on the same test distribution used for the final evaluation. If the latter, the reported improvements may be optimistic.

minor comments (5)

[Table 3, row 'position : price'] The row label 'position : price' is inconsistent with the rest of the paper, which uses 'position : cost'. Please correct the table header or entry.
[Throughout] There are typographical errors: 'Requirment' in Table 1, 'boundinb' in §A.2, and the duplicated 'Pareto-front' in the caption of Figure 5. A thorough proofread is needed.
[Figure 5] The paper contains two Figures numbered 5: one for DPO training curves (§5.2) and one for Pareto fronts (§5.3). Please renumber the figures.
[§5.1, evaluation metric] The normalization of constraint violations to [0,1] and the choice of reference point for hypervolume are not fully specified. Please state the normalization procedure and the reference point used for hypervolume computation.
[References] The reference list contains entries with missing venue information (e.g., 'Lee, H., ...' and 'Zhou, Z., ...'), likely because they were added as preprints. Please add complete publication details where available.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: e-SimFT's Pareto-front claim rests on an empirical comparison against external baselines; the only notable self-citation (GearFormer) supplies the experimental stack but does not force the result.

full rationale

The central claim is an empirical comparison: Tables 2 and 3 report hypervolumes of Pareto fronts from e-SimFT versus the pretrained baseline, Rewarded Soup, Rewards-in-Context, and ablations. RS and RiC are externally developed multi-objective alignment methods, so the comparison has independent content even though all methods are evaluated on the same GearFormer simulator. The use of simulator feedback both for fine-tuning (rewards/preference labels) and for the evaluation metric does not make the result circular: the training objective (e.g., satisfy a cost bound) is not the same object as the measured hypervolume, and the method could in principle perform worse than the baselines. The one notable self-reference is GearFormer (Etesam et al. 2024), whose authors overlap with this paper and which supplies the pretrained model, simulator, and dataset; the paper states 'GearFormer was the only work that provided the model, simulator, and dataset required for our experiments.' This is a scope limitation and a dependency on the authors' prior work, but it is not load-bearing circularity: the comparison against RS and RiC would remain meaningful with any base model, and no uniqueness theorem or ansatz is imported from the cited work to force the conclusion. Section 4.3's assumption that a SimFT model can serve as the epsilon-constrained model is an explicit heuristic, not a claim that the output is defined in terms of the input. The 'post hoc' checkpoint selection in Appendix A is model selection, not a fitted parameter renamed as a prediction. Overall, no step reduces to its own input by construction.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim rests on hand-chosen epsilon ranges, KL penalties, and checkpoint selection procedures, plus a heuristic equivalence between epsilon-sampling and the epsilon-constraint method. No new physical entities are introduced.

free parameters (5)

epsilon range for original requirements = [-5, 5]
Hand-chosen grid bounds for perturbing equality-constraint requirement values in epsilon-sampling (Section 5.1).
epsilon range for new requirements = [0, 10]
Hand-chosen grid bounds for perturbing inequality-constraint bound values in epsilon-sampling (Section 5.1).
KL penalty parameter beta = 0.1
Used in both DPO and PPO losses; chosen by hand (Appendix A.2).
Learning rates = 1e-6 (SFT, DPO), 1e-5 (PPO, new-requirement SFT)
Hand-chosen training hyperparameters (Appendix A.2).
Checkpoint selection criterion = best requirement improvement, subject to >=95% valid designs
Post hoc selection of DPO/PPO checkpoints; may use test data and inflates reported numbers if so (Appendix A.2).

assumptions (3)

ad hoc to paper SFT with rejection-sampled data suffices for fine-tuning on original equality-constraint requirements.
Assumed in Section 3.2 to justify using SFT instead of RL for original requirements; the justification is a short argument, not a proof.
ad hoc to paper A model fine-tuned for requirement ri can best enforce ri as a constraint, so sampling from it with perturbed ri values mimics the epsilon-constraint method.
Load-bearing assumption of epsilon-sampling (Section 4.3); no guarantee that perturbed-condition samples are Pareto-optimal.
domain assumption The GearFormer simulator accurately evaluates the four design requirements (speed ratio, position, cost, bounding box volume).
The simulator is used for both training feedback and evaluation; its accuracy is not independently verified (Section 5.1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of e-SimFT: Alignment of Generative Models with Simulation Feedback for Pareto-Front Design Exploration." pith.science (2026). https://pith.science/paper/R2KFUJLP

@misc{pith2026250202628,
  author       = {Pith},
  title        = {Pith review of: e-SimFT: Alignment of Generative Models with Simulation Feedback for Pareto-Front Design Exploration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R2KFUJLP}},
  note         = {Machine review of arXiv:2502.02628}
}

read the original abstract

Deep generative models have recently shown success in solving complex engineering design problems where models predict solutions that address the design requirements specified as input. However, there remains a challenge in aligning such models for effective design exploration. For many design problems, finding a solution that meets all the requirements is infeasible. In such a case, engineers prefer to obtain a set of Pareto optimal solutions with respect to those requirements, but uniform sampling of generative models may not yield a useful Pareto front. To address this gap, we introduce a new framework for Pareto-front design exploration with simulation fine-tuned generative models. First, the framework adopts preference alignment methods developed for Large Language Models (LLMs) and showcases the first application in fine-tuning a generative model for engineering design. The important distinction here is that we use a simulator instead of humans to provide accurate and scalable feedback. Next, we propose epsilon-sampling, inspired by the epsilon-constraint method used for Pareto-front generation with classical optimization algorithms, to construct a high-quality Pareto front with the fine-tuned models. Our framework, named e-SimFT, is shown to produce better-quality Pareto fronts than existing multi-objective alignment methods.

Figures

Figures reproduced from arXiv: 2502.02628 by the authors.

**Figure 1.** Randomly sampling a generative model (for engineering design) may not yield a good Pareto front with respect to the design requirements of interest. Also, an engineer could be interested in a new design requirement that the current model is not conditioned on to generate design solutions. We address these issues with SimFT methods – using simulation feedback to fine-tune a generative model with respect to specific d… view at source ↗

**Figure 2.** SimFT data generation and training methods. 4 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Epsilon-sampling with a SimFT model. The target requirement values for R1 is incremented with ϵi and a SimFT model for R1 is sampled to construct a Pareto front. 5. Experiments We evaluate the performance improvements made by SimFT methods and perform ablation studies to elucidate important aspects of SimFT. We then evaluate e-SimFT against several baselines in generating high-quality Pareto fronts. 5.1. Experiment… view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: DPO improves the percentage of requirements met at the expense of the percentage of valid designs. assigns a score of 1 or -1 if the requirement is met or not. We found that fine-tuning the model with this reward function for 20 epochs, the best accuracies were 62.3% …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 1 canonical work pages

[3]

Controllable prefer- ence optimization: Toward controllable multi-objective alignment

Guo, Y ., Cui, G., Yuan, L., Ding, N., Sun, Z., Sun, B., Chen, H., Xie, R., Zhou, J., Lin, Y ., et al. Controllable prefer- ence optimization: Toward controllable multi-objective alignment. arXiv preprint arXiv:2402.19085,

arXiv
[6]

R., Bishop, C., Hall, E., Carbune, V ., Rastogi, A., et al

Lee, H., Phatale, S., Mansoor, H., Mesnard, T., Ferret, J., Lu, K. R., Bishop, C., Hall, E., Carbune, V ., Rastogi, A., et al. Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with ai feedback. In Forty-first International Conference on Machine Learning. Liu, J., Zhu, Y ., Xiao, K., Fu, Q., Han, X., Yang, W., and Ye, D. Rltf: Reinforceme...

arXiv
[9]

Rewards-in-context: Multi-objective alignment of foundation models with dynamic preference adjust- ment

Yang, R., Pan, X., Luo, F., Qiu, S., Zhong, H., Yu, D., and Chen, J. Rewards-in-context: Multi-objective alignment of foundation models with dynamic preference adjust- ment. arXiv preprint arXiv:2402.10207,

arXiv
[10]

Panacea: Pareto alignment via preference adaptation for llms

Zhong, Y ., Ma, C., Zhang, X., Yang, Z., Chen, H., Zhang, Q., Qi, S., and Yang, Y . Panacea: Pareto alignment via preference adaptation for llms. arXiv preprint arXiv:2402.02030,

arXiv
[11]

Beyond one-preference-fits-all alignment: Multi-objective direct preference optimization

Zhou, Z., Liu, J., Shao, J., Yue, X., Yang, C., Ouyang, W., and Qiao, Y . Beyond one-preference-fits-all alignment: Multi-objective direct preference optimization. In Find- ings of the Association for Computational Linguistics ACL 2024, pp. 10586–10613,

work page 2024
[12]

M., Stiennon, N., Wu, J., Brown, T

Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., and Irving, G. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593,

arXiv 1909
[1971]

B., and Sadigh, D

Hejna, J., Rafailov, R., Sikchi, H., Finn, C., Niekum, S., Knox, W. B., and Sadigh, D. Contrastive prefence learn- ing: Learning from human feedback without rl. arXiv preprint arXiv:2310.13639,

arXiv
[2016]

Etesam, Y ., Cheong, H., Ataei, M., and Jayaraman, P. K. Deep generative model for mechanical system configura- tion design. arXiv preprint arXiv:2409.06016,

arXiv

Show all 12 references

[2020]

Multi-objective reinforcement learning from ai feedback

Williams, M. Multi-objective reinforcement learning from ai feedback. arXiv preprint arXiv:2406.07295,

arXiv
[2022]

Proximal policy optimization algorithms

Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347,

arXiv
[2023]

Rlsf: Reinforce- ment learning via symbolic feedback

Jha, P., Jana, P., Arora, A., and Ganesh, V . Rlsf: Reinforce- ment learning via symbolic feedback. arXiv preprint arXiv:2405.16661,
[2024]

Kto: Model alignment as prospect theoretic optimization

Ethayarajh, K., Xu, W., Muennighoff, N., Jurafsky, D., and Kiela, D. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306,

arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [3]

Controllable prefer- ence optimization: Toward controllable multi-objective alignment

Guo, Y ., Cui, G., Yuan, L., Ding, N., Sun, Z., Sun, B., Chen, H., Xie, R., Zhou, J., Lin, Y ., et al. Controllable prefer- ence optimization: Toward controllable multi-objective alignment. arXiv preprint arXiv:2402.19085,

arXiv

[2] [6]

R., Bishop, C., Hall, E., Carbune, V ., Rastogi, A., et al

Lee, H., Phatale, S., Mansoor, H., Mesnard, T., Ferret, J., Lu, K. R., Bishop, C., Hall, E., Carbune, V ., Rastogi, A., et al. Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with ai feedback. In Forty-first International Conference on Machine Learning. Liu, J., Zhu, Y ., Xiao, K., Fu, Q., Han, X., Yang, W., and Ye, D. Rltf: Reinforceme...

arXiv

[3] [9]

Rewards-in-context: Multi-objective alignment of foundation models with dynamic preference adjust- ment

Yang, R., Pan, X., Luo, F., Qiu, S., Zhong, H., Yu, D., and Chen, J. Rewards-in-context: Multi-objective alignment of foundation models with dynamic preference adjust- ment. arXiv preprint arXiv:2402.10207,

arXiv

[4] [10]

Panacea: Pareto alignment via preference adaptation for llms

Zhong, Y ., Ma, C., Zhang, X., Yang, Z., Chen, H., Zhang, Q., Qi, S., and Yang, Y . Panacea: Pareto alignment via preference adaptation for llms. arXiv preprint arXiv:2402.02030,

arXiv

[5] [11]

Beyond one-preference-fits-all alignment: Multi-objective direct preference optimization

Zhou, Z., Liu, J., Shao, J., Yue, X., Yang, C., Ouyang, W., and Qiao, Y . Beyond one-preference-fits-all alignment: Multi-objective direct preference optimization. In Find- ings of the Association for Computational Linguistics ACL 2024, pp. 10586–10613,

work page 2024

[6] [12]

M., Stiennon, N., Wu, J., Brown, T

Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., and Irving, G. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593,

arXiv 1909

[7] [1971]

B., and Sadigh, D

Hejna, J., Rafailov, R., Sikchi, H., Finn, C., Niekum, S., Knox, W. B., and Sadigh, D. Contrastive prefence learn- ing: Learning from human feedback without rl. arXiv preprint arXiv:2310.13639,

arXiv

[8] [2016]

Etesam, Y ., Cheong, H., Ataei, M., and Jayaraman, P. K. Deep generative model for mechanical system configura- tion design. arXiv preprint arXiv:2409.06016,

arXiv