Pith. sign in

REVIEW 5 cited by

SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.07471 v2 pith:BXY6Z5YI submitted 2024-10-09 cs.LG cs.AIcs.CL

Han Shen , Pin-Yu Chen , Payel Das , Tianyi Chen This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords datafine-tuningsealmodelsbilevelgithubranksafety
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Fine-tuning on task-specific data to boost downstream performance is a crucial step for leveraging Large Language Models (LLMs). However, previous studies have demonstrated that fine-tuning the models on several adversarial samples or even benign data can greatly comprise the model's pre-equipped alignment and safety capabilities. In this work, we propose SEAL, a novel framework to enhance safety in LLM fine-tuning. SEAL learns a data ranker based on the bilevel optimization to up rank the safe and high-quality fine-tuning data and down rank the unsafe or low-quality ones. Models trained with SEAL demonstrate superior quality over multiple baselines, with 8.5% and 9.7% win rate increase compared to random selection respectively on Llama-3-8b-Instruct and Merlinite-7b models. Our code is available on github https://github.com/hanshen95/SEAL.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Sharper Analysis of Single-Loop Methods for Bilevel Optimization

    cs.LG 2026-07 accept novelty 6.0 of 10

    Decoupled-norm analysis improves single-loop AID to O(κ⁵/K) and shows single-loop ITD's asymptotic error is exactly O(κ²), matching the known lower bound.

  2. DeFrame: Debiasing Large Language Models Against Framing Effects

    cs.CL 2026-02 conditional novelty 6.0 of 10

    LLM fairness scores shift substantially with positive vs negative framing of the same question, and DeFrame—a three-step self-revision prompt—reduces both average bias and this framing gap.

  3. BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining

    cs.LG 2025-10 conditional novelty 6.0 of 10

    A bilevel optimization method ranks pretraining data by training a small proxy model on weighted samples, yielding modest downstream-task gains without external pretrained models.

  4. A Survey on Open Dataset Search in the LLM Era: Retrospectives and Perspectives

    cs.IR 2025-08 conditional novelty 4.0 of 10

    A structured review of open dataset search across tabular, spatial, JSON, graph, and vector data, plus the two-way relationship with LLMs.

  5. Bayesian Optimization for Enhanced Language Models: Optimizing Acquisition Functions

    cs.CL 2025-05 reject novelty 4.0 of 10

    A bilevel Bayesian optimization strategy pairing EI and UCB acquisition functions improves RoBERTa fine-tuning by up to 2.7 percent on a subset of GLUE tasks, but the results lack statistical validation.

Pith tools