Pith. sign in

REVIEW 20 cited by

Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2311.10702 v2 pith:E6M3GHPR submitted 2023-11-17 cs.CL

Hamish Ivison , Yizhong Wang , Valentina Pyatkin , Nathan Lambert , Matthew Peters , Pradeep Dasigi , Joel Jang , David Wadden
show 3 more authors
Noah A. Smith Iz Beltagy Hannaneh Hajishirzi
This is my paper · ORCID
classification cs.CL
keywords modelscodeopenreleaseadaptingevaluationfinetunedimproved
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Since the release of T\"ULU [Wang et al., 2023b], open resources for instruction tuning have developed quickly, from better base models to new finetuning techniques. We test and incorporate a number of these advances into T\"ULU, resulting in T\"ULU 2, a suite of improved T\"ULU models for advancing the understanding and best practices of adapting pretrained language models to downstream tasks and user preferences. Concretely, we release: (1) T\"ULU-V2-mix, an improved collection of high-quality instruction datasets; (2) T\"ULU 2, LLAMA-2 models finetuned on the V2 mixture; (3) T\"ULU 2+DPO, T\"ULU 2 models trained with direct preference optimization (DPO), including the largest DPO-trained model to date (T\"ULU 2+DPO 70B); (4) CODE T\"ULU 2, CODE LLAMA models finetuned on our V2 mix that outperform CODE LLAMA and its instruction-tuned variant, CODE LLAMA-Instruct. Our evaluation from multiple perspectives shows that the T\"ULU 2 suite achieves state-of-the-art performance among open models and matches or exceeds the performance of GPT-3.5-turbo-0301 on several benchmarks. We release all the checkpoints, data, training and evaluation code to facilitate future open efforts on adapting large language models.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 20 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs

    cs.CL 2025-07 conditional novelty 7.0 of 10

    Cognitive biases in LLMs are largely set during pretraining, while finetuning data and seed randomness only modulate them.

  2. A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)

    cs.LG 2026-02 conditional novelty 6.0 of 10

    Only gradient-based (LESS) representations make subset-to-query distance a reliable predictor of instruction-tuning performance; greedy round-robin helps most at small budgets, and random selection is surprisingly com...

  3. Less is Enough: Synthesizing Diverse Data in LLM Feature Space with Sparse Autoencoders

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Coverage of sparse-autoencoder-identified task features predicts post-training performance and can guide synthesis of small, high-impact datasets (2,000 vs. 300,000 samples).

  4. Select or Project? Evaluating Lower-dimensional Vectors for LLM Training Data Explanations

    cs.CL 2026-01 reject novelty 6.0 of 10

    A greedy gradient-component selection appears to beat full gradients and random projection for training-data retrieval, but the comparison is confounded because components are selected on the evaluation set itself.

  5. DPO Unchained: Your Training Algorithm is Secretly Disentangled in Human Choice Theory (and its Loss' Convexity is Dispensable)

    cs.LG 2025-07 conditional novelty 6.0 of 10

    DPO's connection to human choice theory generalizes: any monotone loss can pair with any choice model in a new KLST* class, so loss convexity is not normatively required.

  6. On the Effect of Instruction Tuning Loss on Generalization

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Weighted Instruction Tuning, with low-to-moderate prompt weight and moderate-to-high response weight, beats the standard response-only instruction tuning loss in most of the 75 (model, dataset, benchmark) settings tested.

  7. Disentangling the Roles of Representation and Selection in Data Pruning

    cs.CL 2025-07 conditional novelty 6.0 of 10

    In NLP data pruning, the representation used to score examples (especially gradients) influences selected data and downstream performance more than the selection algorithm, and difficulty-oriented algorithms often do ...

  8. Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    The monotonicity of token probabilities during initial decoding predicts chain-of-thought gains, enabling dynamic selection between CoT and direct answers.

  9. Data Swarms: Optimizable Generation of Synthetic Evaluation Data

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Data Swarms uses particle swarm optimization over data-generator LLM weights to produce synthetic evaluation data that scores higher on five quantitative evaluation objectives than eight baselines.

  10. Systematic Evaluation of Machine-Generated Reasoning and PHQ-9 Labeling for Depression Detection Using Large Language Models

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A subtask decomposition of depression detection shows LLMs are biased by explicit depression keywords, and DPO fine-tuning on quality-filtered machine-generated rationales improves joint PHQ-9 labeling on the hardest samples.

  11. IHEval: Evaluating Language Models on Following the Instruction Hierarchy

    cs.CL 2025-02 conditional novelty 6.0 of 10

    IHEval shows that current language models often follow lower-priority instructions over system messages, and simple prompting does not fix the problem.

  12. SHARP: Accelerating Language Model Inference by SHaring Adjacent layers with Recovery Parameters

    cs.LG 2025-02 conditional novelty 6.0 of 10

    Sharing one MLP layer's weights across several layers plus low-rank adapters recovers most of a pretrained LLM's quality with a fraction of the storage and faster phone inference.

  13. From Black Box to Transparency: Enhancing Automated Interpreting Assessment with Explainable AI in College Classrooms

    cs.CL 2025-08 unverdicted novelty 5.0 of 10

    An explainable model using BLEURT, CometKiwi, pause features, and Chinese phraseological diversity predicts human-rated quality dimensions in English-Chinese consecutive interpreting, with SHAP identifying the stronge...

  14. Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Nile-Chat models for dual-script Egyptian Arabic beat strong baselines on newly translated benchmarks, but the evaluation may be inflated by training/eval data overlap and Claude-generated script data.

  15. AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMs

    cs.LG 2025-05 conditional novelty 5.0 of 10

    AutoMixAlign adaptively reweights or resamples task data during DPO training to match specialist-model losses, improving average performance on helpfulness, coding, and safety benchmarks compared to standard DPO and m...

  16. Efficient Data Selection at Scale via Influence Distillation

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Influence Distillation selects LLM fine-tuning data by approximating each sample's gradient influence on a target task via landmarks and JVP embeddings, matching or beating RDS+ accuracy at roughly one third the selec...

  17. Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation

    cs.CV 2025-08 conditional novelty 4.0 of 10

    A hierarchical direct preference optimization with four alignment levels plus automated data selection improves physical plausibility of text-to-video models.

  18. DiffLoRA: Differential Low-Rank Adapters for Large Language Models

    cs.CL 2025-07 conditional novelty 4.0 of 10

    DiffLoRA, a low-rank adapter variant of differential attention, generally underperforms LoRA but shows gains on HumanEval and multi-value needle retrieval.

  19. Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting

    cs.CL 2025-06 conditional novelty 4.0 of 10

    A self-generated and committee-filtered synthetic rehearsal dataset lets a fine-tuned LLM nearly preserve its general benchmark scores while adding domain data.

  20. Why Do More Experts Fail? A Theoretical Analysis of Model Merging

    cs.LG 2025-05 reject novelty 4.0 of 10

    The paper claims to prove an upper bound and diminishing returns in model merging, but the proofs are not sound and the heavy-tailed claim is contradicted by its own equations.

Pith tools