Paper Citation Record · LEDGER

A General Theoretical Paradigm to Understand Learning from Human Preferences

As of 13 August 2026, this Paper Citation Record lists 0 of 0 outbound references and 45 inbound Pith citation observations for arXiv:2310.12036.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2310.12036 v2

Coverage vector

measured 0 of 0 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links

measured 45 of 45 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-12T06:34:41.77262+00:00

measured 45 of 45 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-12T17:05:15.089359Z

measured 1 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-08-05T02:28:24.338817Z

Reference resolution

0 of 0 outbound references displayed

verified exact0
verified fuzzy0
unresolved0
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

14
arxiv_reference, observed 2026-08-05T02:28:24.338817Z

Outbound references

No outbound reference observations are available for this paper version.

Pith citing papers

Observation b045bfd9-05d4-4f25-8c19-72a780e85c46 · inbound

ORPO: Monolithic Preference Optimization without Reference Model cites this paper.

ORPO: Monolithic Preference Optimization without Reference Model A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 3

Resolution: verified exact; arxiv_id, observed 2026-05-16T09:34:04.769240Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 42e8f3a9-a770-4e32-93e9-97381c0226f8 · inbound

DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs cites this paper.

DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-12T17:05:15.089359Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 267afe4a-96dc-445d-801f-0d2a3660b55c · inbound

Free Process Rewards without Process Labels cites this paper.

Free Process Rewards without Process Labels A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-12T00:06:17.842642Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 15dea84b-555b-4a49-98f8-50a8e061c7ca · inbound

Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model cites this paper.

Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-11T12:48:58.822647Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a6df2947-5ffe-4330-ab02-943c86d44356 · inbound

Understanding the Logic of Direct Preference Alignment through Logic cites this paper.

Understanding the Logic of Direct Preference Alignment through Logic A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 2020

Resolution: unresolved; no resolver link, observed 2026-08-11T05:24:20.892964Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d8f7dbff-b358-418f-935e-0c88108c1415 · inbound

InfAlign: Inference-aware language model alignment cites this paper.

InfAlign: Inference-aware language model alignment A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-10T23:57:54.202526Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 271cf759-74ff-4e8c-bbca-483ebcf9d94b · inbound

Enhanced Vision-Language Models for Diverse Sensor Understanding: Cost-Efficient Optimization and Benchmarking cites this paper.

Enhanced Vision-Language Models for Diverse Sensor Understanding: Cost-Efficient Optimization and Benchmarking A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 70

Resolution: unresolved; no resolver link, observed 2026-08-10T23:16:55.356697Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5aaa6f2a-ec60-4d7e-bc62-efde2d93a408 · inbound

The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hacking cites this paper.

The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hacking A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-09T20:34:01.215406Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7699aa7b-11f8-4f6e-8770-38fe3745ab72 · inbound

On Almost Surely Safe Alignment of Large Language Models at Inference-Time cites this paper.

On Almost Surely Safe Alignment of Large Language Models at Inference-Time A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-09T16:18:40.577727Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 972de14a-91e6-49d4-94cb-6ccdaf5c574b · inbound

Process Reinforcement through Implicit Rewards cites this paper.

Process Reinforcement through Implicit Rewards A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 5

Resolution: verified exact; arxiv_id, observed 2026-05-11T20:23:30.944982Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation cf191c35-876d-4119-a643-6416b43f25a9 · inbound

On Fairness of Unified Multimodal Large Language Model for Image Generation cites this paper.

On Fairness of Unified Multimodal Large Language Model for Image Generation A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 2023

Resolution: unresolved; no resolver link, observed 2026-08-09T04:52:40.534383Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b3153567-2340-453f-ad62-cc1375bc3e29 · inbound

LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions cites this paper.

LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 2023

Resolution: unresolved; no resolver link, observed 2026-08-07T15:07:59.145850Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 54a2c4d8-ace3-40d0-979d-4e89e722de36 · inbound

OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Models cites this paper.

OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Models A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 129

Resolution: unresolved; no resolver link, observed 2026-08-07T14:31:45.997262Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f93d914f-0c4c-473a-82e8-2c013645e3ae · inbound

Thompson Sampling in Online RLHF with General Function Approximation cites this paper.

Thompson Sampling in Online RLHF with General Function Approximation A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-07T12:43:47.923648Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c9abefb6-8aaf-4792-96b9-8cd2dca89181 · inbound

Multi-objective Aligned Bidword Generation Model for E-commerce Search Advertising cites this paper.

Multi-objective Aligned Bidword Generation Model for E-commerce Search Advertising A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-07T10:57:42.228327Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 38a7ec8d-1f7a-4786-babf-f21180eb9c63 · inbound

Reinforce LLM Reasoning through Multi-Agent Reflection cites this paper.

Reinforce LLM Reasoning through Multi-Agent Reflection A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 2023

Resolution: unresolved; no resolver link, observed 2026-08-07T05:22:07.333606Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b2c59bc0-5896-4dd9-aef2-605091779e68 · inbound

DETONATE: A Benchmark for Text-to-Image Alignment and Kernelized Direct Preference Optimization cites this paper.

DETONATE: A Benchmark for Text-to-Image Alignment and Kernelized Direct Preference Optimization A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-07T00:16:06.358214Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fe1636fe-aa28-4d0a-a15d-841dcc051593 · inbound

CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation cites this paper.

CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 50

Resolution: unresolved; no resolver link, observed 2026-08-06T19:18:40.283348Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e1575096-4be4-46e7-b026-bd9c440d36d0 · inbound

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap cites this paper.

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 31

Resolution: verified exact; arxiv_id, observed 2026-05-21T23:50:47.607776Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation fec1937c-8fd6-4e10-b526-f01cd2d18c30 · inbound

Enhancing Small LLM Alignment through Margin-Based Objective Modifications under Resource Constraints cites this paper.

Enhancing Small LLM Alignment through Margin-Based Objective Modifications under Resource Constraints A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-05T21:33:13.809159Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4e7e3696-975a-4dd5-9d5b-d36ed9ffd8d3 · inbound

Failure Modes of Maximum Entropy RLHF cites this paper.

Failure Modes of Maximum Entropy RLHF A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 7

Resolution: verified exact; arxiv_id, observed 2026-05-18T14:02:39.813225Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 4deb9cf5-bfb4-4a4e-a885-a13706c67041 · inbound

Adaptive Margin RLHF via Preference over Preferences cites this paper.

Adaptive Margin RLHF via Preference over Preferences A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-04T14:52:18.668425Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d058d0b3-5f42-4060-9235-c76e24e0b6db · inbound

Safety Alignment of LMs via Non-cooperative Games cites this paper.

Safety Alignment of LMs via Non-cooperative Games A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-03T14:24:19.502646Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 03710aaf-d50c-4e56-b8a4-670bcc2e4707 · inbound

Gradient-Gated DPO: Stabilizing Preference Optimization in Language Models cites this paper.

Gradient-Gated DPO: Stabilizing Preference Optimization in Language Models A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 1

Resolution: verified exact; arxiv_id, observed 2026-05-09T06:20:41.683804Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation ae04bdcb-ef5f-4f03-9cd2-af5fee1a1754 · inbound

Response Time Enhances Alignment with Heterogeneous Preferences cites this paper.

Response Time Enhances Alignment with Heterogeneous Preferences A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 4

Resolution: verified exact; arxiv_id, observed 2026-05-11T01:05:50.004199Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 2deb40d5-6e9a-486a-9ba4-13dc4827f5c1 · inbound

Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Models cites this paper.

Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Models A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 5

Resolution: verified exact; arxiv_id, observed 2026-05-13T07:07:28.007161Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 36ad412e-a310-41da-b084-7a7d3f8d7d7d · inbound

Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Models cites this paper.

Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Models A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 5

Resolution: verified exact; arxiv_id, observed 2026-05-14T21:19:28.313009Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 89c015fc-013f-4f68-85d5-8f127abb6416 · inbound

YFPO: A Preliminary Study of Yoked Feature Preference Optimization with Neuron-Guided Rewards for Mathematical Reasoning cites this paper.

YFPO: A Preliminary Study of Yoked Feature Preference Optimization with Neuron-Guided Rewards for Mathematical Reasoning A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 22

Resolution: verified exact; arxiv_id, observed 2026-05-13T05:32:19.327417Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 4b7e7949-a768-47e2-85a4-642bbdd8a483 · inbound

TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching cites this paper.

TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 136

Resolution: verified exact; arxiv_id, observed 2026-05-13T04:57:17.181807Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 09a58498-7564-4110-a7c0-603eedba08a4 · inbound

TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching cites this paper.

TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 136

Resolution: verified exact; arxiv_id, observed 2026-05-15T05:45:06.641433Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation ce4112a6-da47-454b-a8cf-6dbba167bf8f · inbound

CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models cites this paper.

CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 1

Resolution: verified exact; arxiv_id, observed 2026-05-22T08:11:17.214389Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 735f9d5d-4baf-4dcc-be85-c42485d78836 · inbound

CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models cites this paper.

CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 1

Resolution: verified exact; arxiv_id, observed 2026-07-01T15:05:47.998440Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 5b41e8e6-8bf3-4891-842a-8fd6f1a8a774 · inbound

AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates cites this paper.

AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 2

Resolution: verified exact; arxiv_id, observed 2026-06-29T12:33:24.402718Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation bb7add3f-651c-4f56-bc78-9cd92d032cec · inbound

Constitutional On-Policy Safe Distillation cites this paper.

Constitutional On-Policy Safe Distillation A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 2

Resolution: verified exact; arxiv_id, observed 2026-07-02T01:36:25.512570Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 75c04942-f7a4-49e3-81b1-6bda7cfbccb3 · inbound

FlowPRO: Reward-Free Reinforced Fine-Tuning of Flow-Matching VLAs via Proximalized Preference Optimization cites this paper.

FlowPRO: Reward-Free Reinforced Fine-Tuning of Flow-Matching VLAs via Proximalized Preference Optimization A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 29

Resolution: verified exact; arxiv_id, observed 2026-07-02T09:06:49.361855Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 5b049f08-6e51-467c-9c92-342a0f7eb1a2 · inbound

Weight-Space Geometry of Offline Reasoning Training cites this paper.

Weight-Space Geometry of Offline Reasoning Training A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 14

Resolution: verified exact; arxiv_id, observed 2026-07-04T09:09:43.385341Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation a11da4f5-e823-4ed4-a425-56b5519e8320 · inbound

Towards Spec Learning: Inference-Time Alignment from Preference Pairs cites this paper.

Towards Spec Learning: Inference-Time Alignment from Preference Pairs A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 4

Resolution: verified exact; arxiv_id, observed 2026-07-04T11:39:47.161717Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 38364724-f099-488a-a77c-71f5162f8b76 · inbound

Towards Spec Learning: Inference-Time Alignment from Preference Pairs cites this paper.

Towards Spec Learning: Inference-Time Alignment from Preference Pairs A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 4

Resolution: verified exact; arxiv_id, observed 2026-06-30T12:44:39.556729Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation a1437ddd-5690-4ed6-a8bc-fa94eab0e2ef · inbound

The Hitchhiker's Guide to Agentic AI: From Foundations to Systems cites this paper.

The Hitchhiker's Guide to Agentic AI: From Foundations to Systems A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 13

Resolution: verified exact; arxiv_id, observed 2026-07-04T11:09:46.374927Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation b6fc0ef8-98a3-4acf-9eb1-8386c9e100f9 · inbound

The Hitchhiker's Guide to Agentic AI: From Foundations to Systems cites this paper.

The Hitchhiker's Guide to Agentic AI: From Foundations to Systems A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-02T10:27:16.169555Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a5e8bc7a-5eba-4e50-a3b7-20421d5bdd65 · inbound

ACPO: Adaptive Credit Policy Optimization via Fine-Grained Surrogate Entropy cites this paper.

ACPO: Adaptive Credit Policy Optimization via Fine-Grained Surrogate Entropy A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 67

Resolution: unresolved; no resolver link, observed 2026-07-12T04:43:45.592808Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a8fe755a-582d-4e88-b4ca-db7b229fe51b · inbound

Multi-Turn On-Policy Distillation with Prefix Replay cites this paper.

Multi-Turn On-Policy Distillation with Prefix Replay A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 163

Resolution: unresolved; no resolver link, observed 2026-07-11T13:53:36.775836Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b839b03e-a437-4a0d-a884-49efdc099734 · inbound

Multi-Turn On-Policy Distillation with Prefix Replay cites this paper.

Multi-Turn On-Policy Distillation with Prefix Replay A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 164

Resolution: unresolved; no resolver link, observed 2026-08-02T08:40:51.116483Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8899f132-d89f-4e05-bac8-94441a09aef5 · inbound

(Towards) Scalable Reliable Automated Evaluation with Large Language Models cites this paper.

(Towards) Scalable Reliable Automated Evaluation with Large Language Models A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 40

Resolution: unresolved; no resolver link, observed 2026-07-31T12:20:07.021635Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a51d4882-beae-4871-b73f-73d1a086b0f0 · inbound

Quo Vadis, World Modeling? cites this paper.

Quo Vadis, World Modeling? A General Theoretical Paradigm to Understand Learning from Human Preferences

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-07T00:13:50.414984Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.