Pith. sign in

REVIEW 8 cited by

OpenChat: Advancing Open-source Language Models with Mixed-Quality Data

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2309.11235 v2 pith:IZSYSQ2Q submitted 2023-09-20 cs.CL

Guan Wang , Sijie Cheng , Xianyuan Zhan , Xiangang Li , Sen Song , Yang Liu This is my paper · ORCID
classification cs.CL
keywords datamodelsopenchatlanguageopen-sourcepreferencerlftc-rlft
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Nowadays, open-source large language models like LLaMA have emerged. Recent developments have incorporated supervised fine-tuning (SFT) and reinforcement learning fine-tuning (RLFT) to align these models with human goals. However, SFT methods treat all training data with mixed quality equally, while RLFT methods require high-quality pairwise or ranking-based preference data. In this study, we present a novel framework, named OpenChat, to advance open-source language models with mixed-quality data. Specifically, we consider the general SFT training data, consisting of a small amount of expert data mixed with a large proportion of sub-optimal data, without any preference labels. We propose the C(onditioned)-RLFT, which regards different data sources as coarse-grained reward labels and learns a class-conditioned policy to leverage complementary data quality information. Interestingly, the optimal policy in C-RLFT can be easily solved through single-stage, RL-free supervised learning, which is lightweight and avoids costly human preference labeling. Through extensive experiments on three standard benchmarks, our openchat-13b fine-tuned with C-RLFT achieves the highest average performance among all 13b open-source language models. Moreover, we use AGIEval to validate the model generalization performance, in which only openchat-13b surpasses the base model. Finally, we conduct a series of analyses to shed light on the effectiveness and robustness of OpenChat. Our code, data, and models are publicly available at https://github.com/imoneoi/openchat and https://huggingface.co/openchat.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VisCon-100K: Leveraging Contextual Web Data for Fine-tuning Vision Language Models

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A web-context-derived dataset and a 'leaky modality mix' of captions with Q&A pairs improve vision-language model fine-tuning on several benchmarks.

  2. ViewSRD: 3D Visual Grounding via Structured Multi-View Decomposition

    cs.CV 2025-07 conditional novelty 5.0 of 10

    ViewSRD improves 3D visual grounding by decomposing multi-anchor language queries and adding learned view tokens to align text and point clouds across perspectives.

  3. Evaluating LLMs Robustness in Less Resourced Languages with Proxy Models

    cs.CL 2025-06 conditional novelty 5.0 of 10

    Important words chosen by a small proxy model, when perturbed with typos or spacing errors, push Bielik, Mistral-7B, and Llama-3.1-8B to wrong answers on Polish classification tasks more often than random edits.

  4. SciGPT: A Large Language Model for Scientific Literature Understanding and Knowledge Discovery

    cs.CL 2025-09 reject novelty 4.0 of 10

    SciGPT, a fine-tuned Qwen3 model for scientific literature, is reported to outperform GPT-4 on a new ScienceBench benchmark, but the evaluation is unreliable due to missing artifacts and contradictory numbers.

  5. GuardVal: Dynamic Large Language Model Jailbreak Evaluation for Comprehensive Safety Testing

    cs.LG 2025-07 reject novelty 4.0 of 10

    GuardVal combines role-playing jailbreak generation with an Adam-inspired optimizer and an Overall Safety Value metric, but the method is underspecified and not validated with released code or data.

  6. EvoMoE: Expert Evolution in Mixture of Experts for Multimodal Large Language Models

    cs.CL 2025-05 conditional novelty 4.0 of 10

    EvoMoE creates MoE experts as decaying averages of a single trained FFN and routes tokens with hypernetwork-generated weights, yielding small benchmark gains over MoE-LLaVA.

  7. LLM Web Dynamics: Tracing Model Collapse in a Network of LLMs

    cs.LG 2025-05 conditional novelty 4.0 of 10

    Under a shared retrieval-augmented memory, multiple LLMs' outputs converge to near-identical semantic answers, and the analogous Gaussian mixture system is proven to collapse.

  8. OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Models

    cs.CY 2025-05 conditional novelty 4.0 of 10

    The paper advocates protecting and leveraging OpenReview's peer review corpus as a community asset for LLM-based review assistance, benchmarks, and alignment.

Pith tools