REVIEW 5 cited by
sDPO: Don't Use Your Data All at Once
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
As development of large language models (LLM) progresses, aligning them with human preferences has become increasingly important. We propose stepwise DPO (sDPO), an extension of the recently popularized direct preference optimization (DPO) for alignment tuning. This approach involves dividing the available preference datasets and utilizing them in a stepwise manner, rather than employing it all at once. We demonstrate that this method facilitates the use of more precisely aligned reference models within the DPO training framework. Furthermore, sDPO trains the final model to be more performant, even outperforming other popular LLMs with more parameters.
Forward citations
Cited by 5 Pith papers
-
Test-Time Scaling via Error Localization
TTEL uses feedback-induced token probability drops to localize the first error in a failed reasoning trace and branch a new generation from that prefix, improving pass@k per token on coding and math benchmarks.
-
TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization
A fast flow-matching text-to-audio model aligned via CLAP-ranked self-generated preference pairs reports state-of-the-art AudioCaps and human-evaluation scores.
-
CTR-Driven Ad Text Generation via Online Feedback Preference Optimization
CTOP combines retrieval-augmented style transfer with DPO weighted by CTR gain and AA-group confidence, achieving +4.76% relative CTR over human-written ad titles in online tests.
-
PerPO: Perceptual Preference Optimization via Discriminative Rewarding
PerPO trains multimodal LLMs by ranking their candidate answers with deterministic visual rewards (IoU, edit distance) and using the reward differences as margins in listwise preference optimization.
-
An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problems
A survey and position paper that reviews LLM prompting, RAG, and RL techniques and argues they could support open-ended implementation generation, without presenting new results.
Discussion (0). Continue with ORCID to comment.