Paper Citation Record · LEDGER

Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

As of 18 August 2026, this Paper Citation Record lists 0 of 0 outbound references and 19 inbound Pith citation observations for arXiv:2405.16436.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2405.16436 v3

Coverage vector

measured 0 of 0 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links

measured 19 of 19 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-18T06:34:40.430872+00:00

measured 19 of 19 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-15T23:58:38.580544Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-05-13T15:51:29.355251Z

Reference resolution

0 of 0 outbound references displayed

verified exact0
verified fuzzy0
unresolved0
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

No outbound reference observations are available for this paper version.

Pith citing papers

Observation 3d1462c9-e3e2-48ea-acd6-4e4ea29ce460 · inbound

DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs cites this paper.

DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-12T17:05:15.149077Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a30925e4-dab7-415a-a62f-7d9533e6f59d · inbound

Continual SFT Matches Multimodal RLHF with Negative Supervision cites this paper.

Continual SFT Matches Multimodal RLHF with Negative Supervision Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-12T14:58:42.993918Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f421e724-8314-48ab-9ee7-d48c232a2812 · inbound

Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability cites this paper.

Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-12T05:43:31.667639Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4ecb5d6a-a9cb-44b6-a322-befa3abcb542 · inbound

Self-Improvement in Language Models: The Sharpening Mechanism cites this paper.

Self-Improvement in Language Models: The Sharpening Mechanism Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 29

Resolution: unresolved; no resolver link, observed 2026-08-12T00:11:14.724462Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0ab72e31-c006-48d2-a228-b9f56114878b · inbound

Teaching LLMs to Refine with Tools cites this paper.

Teaching LLMs to Refine with Tools Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 2023

Resolution: unresolved; no resolver link, observed 2026-08-11T06:05:31.165998Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5370dfe7-0ac7-412f-ac8c-257b3516106c · inbound

Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs cites this paper.

Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 267

Resolution: verified exact; arxiv_id, observed 2026-05-13T15:51:29.358597Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation ec8aa77e-c5a3-47c4-b3f7-b06b5ca6f11f · inbound

AlphaPO: Reward Shape Matters for LLM Alignment cites this paper.

AlphaPO: Reward Shape Matters for LLM Alignment Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-10T21:51:08.274860Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9bef8d91-24eb-4dc5-ba50-9ed4ec2f475a · inbound

BRiTE: Bootstrapping Reinforced Thinking Process to Enhance Language Model Reasoning cites this paper.

BRiTE: Bootstrapping Reinforced Thinking Process to Enhance Language Model Reasoning Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-09T22:20:13.742905Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation df2d7f18-845b-4a78-b1e6-f792ebd39473 · inbound

Preference Optimization via Contrastive Divergence: Your Reward Model is Secretly an NLL Estimator cites this paper.

Preference Optimization via Contrastive Divergence: Your Reward Model is Secretly an NLL Estimator Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-08T22:27:11.479611Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8e476658-38ac-4c5e-8c3c-b49441490b4c · inbound

Design Considerations in Offline Preference-based RL cites this paper.

Design Considerations in Offline Preference-based RL Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-08T19:40:42.009598Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b136f837-6d43-4c34-937a-3bc5eb491de4 · inbound

DPO-Shift: Shifting the Distribution of Direct Preference Optimization cites this paper.

DPO-Shift: Shifting the Distribution of Direct Preference Optimization Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-08T12:17:45.923786Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9e5458fd-c9ec-4ed2-9613-0ef9f29fe47b · inbound

Policy-labeled Preference Learning: Is Preference Enough for RLHF? cites this paper.

Policy-labeled Preference Learning: Is Preference Enough for RLHF? Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-15T23:58:38.580544Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 206d38f8-6001-4688-a32f-53524120eb29 · inbound

Square$\chi$PO: Differentially Private and Robust $\chi^2$-Preference Optimization in Offline Direct Alignment cites this paper.

Square$\chi$PO: Differentially Private and Robust $\chi^2$-Preference Optimization in Offline Direct Alignment Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 46

Resolution: unresolved; no resolver link, observed 2026-08-07T13:44:57.905368Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 551e7d34-90c3-4fd9-adf5-b9717b571467 · inbound

LeanPO: Lean Preference Optimization for Likelihood Alignment in Video-LLMs cites this paper.

LeanPO: Lean Preference Optimization for Likelihood Alignment in Video-LLMs Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-07T10:28:47.390690Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ae148c67-3aee-4709-bed9-cfe1c368eacc · inbound

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment cites this paper.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:38.224309Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4b856fd7-88d8-4400-897b-592296f9adef · inbound

Robust Single-Stage Fully Sparse 3D Object Detection via Detachable Latent Diffusion cites this paper.

Robust Single-Stage Fully Sparse 3D Object Detection via Detachable Latent Diffusion Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 39

Resolution: unresolved; no resolver link, observed 2026-08-06T04:36:13.529125Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 72162c4c-fb81-48bd-96d3-f3e60f6772bc · inbound

Gradient-Gated DPO: Stabilizing Preference Optimization in Language Models cites this paper.

Gradient-Gated DPO: Stabilizing Preference Optimization in Language Models Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 8

Resolution: verified exact; arxiv_id, observed 2026-05-09T06:20:41.658973Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 1a6b039b-9246-4efc-8272-be4ed36d7e4b · inbound

Multi-Turn On-Policy Distillation with Prefix Replay cites this paper.

Multi-Turn On-Policy Distillation with Prefix Replay Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 87

Resolution: unresolved; no resolver link, observed 2026-07-11T13:53:36.775836Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 779f7c7f-44bf-413e-bb64-6a5a6ac6338b · inbound

Multi-Turn On-Policy Distillation with Prefix Replay cites this paper.

Multi-Turn On-Policy Distillation with Prefix Replay Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 88

Resolution: unresolved; no resolver link, observed 2026-08-02T08:40:41.047599Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.