Pith. sign in

REVIEW 3 major objections 4 minor 5 cited by

SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SelfCite lifts LLM citation F1 by 5.3 points with no human labels

desk verdict A clever, well-ablated self-supervised citation reward that deserves review, but the headline F1 gain needs an error-bar and baseline-reconciliation pass. read the letter →

arxiv 2502.09604 v3 pith:Q6VUNR6W submitted 2025-02-13 cs.CL cs.AIcs.LG

Yung-Sung Chuang , Benjamin Cohen-Wang , Shannon Zejiang Shen , Zhaofeng Wu , Hu Xu , Xi Victoria Lin , James Glass , Shang-Wen Li
show 1 more author
Wen-tau Yih
This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords self-supervisedalignmentcontextattributioncitationsLongBench-Citeablationbest-of-NsamplingpreferenceoptimizationSimPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that citation quality in long-form question answering can be improved substantially using only the model’s own output probabilities, with no human or API annotations during the alignment stage. Its core idea is a self-supervised reward computed by context ablation: removing the cited sentences should make the response less probable (necessity), and keeping only the cited sentences should preserve it (sufficiency). Applied as best-of-N reranking over candidate citation spans, and then distilled into the model with SimPO preference optimization, the reward raises citation F1 by up to 5.3 points on LongBench-Cite across five long-context QA tasks. A fully self-supervised variant, starting from a model whose citations were generated by ContextCite rather than human data, also improves. If correct, the result means citation quality can be bootstrapped without expensive annotation, and smaller open models can approach proprietary citation pipelines.

What carries the argument

The load-bearing object is the context-ablation reward pair: a necessity score (probability drop when the cited sentences are removed) plus a sufficiency score (probability hold when only the cited sentences remain), which telescope into Eq. (1). This reward is computed with the language model in eval mode, requires no trained reward model or annotation, and is used in two ways: (i) best-of-N sampling reranks N citation-span candidates per statement by the reward; (ii) preference optimization with SimPO uses the BoN-improved responses as chosen examples to internalize the behavior. The token-length cap (384 tokens unless the citation is a single sentence) prevents the model from gaming the reward by citing long passages.

What would settle it

Measure, on a held-out set of LongBench-Cite statements with human citation-quality judgments, the rank correlation between the SelfCite reward and human citation precision/recall: if the reward's ranking of alternative citation candidates does not positively correlate with human scores (or if inserting an irrelevant but causally influential sentence into a citation increases the reward), the central claim that the ablation reward tracks citation quality is falsified.

Watch

Extended reading notes

Core claim

The central discovery is that the difference between the log-probability of a response statement given only the cited sentences and given the context minus the cited sentences—$\text{Reward}(e_i) = \log p_{LM}(r_i \mid E_i) - \log p_{LM}(r_i \mid C \setminus E_i)$—is a sufficient reward signal to rerank candidate citations. The paper shows that selecting, for each statement, the candidate citation that maximizes this reward (with a token-length cap) improves both citation recall and precision, pushing average citation F1 from 73.8 to 77.5 on LongBench-Cite. Training with SimPO on preference pairs built from best-of-N selections matches this gain in a single forward pass (77.9 F1), and applying best-of-N again on the SimPO model reaches 79.1 F1, an improvement of 5.3 points over the base model. The authors interpret this as evidence that the contributive, causality-oriented attribution signal is at least partially aligned with the corroborative, support-based citation quality measured by the benchmark.

Load-bearing premise

The method assumes that the change in the model's own next-token probability when context is ablated faithfully tracks whether a citation is necessary and sufficient, and that this contributive signal improves the external, corroborative citation F1 score measured by GPT-4o on LongBench-Cite; it also assumes the base model can already generate parseable citation candidates.

Editorial extensions

If this is right

  • Citation alignment can be performed without human annotation or proprietary APIs, since the reward is computed from the model's own probabilities.
  • Best-of-N sampling alone yields immediate citation gains (73.8 to 77.5 F1) at inference time, with only short citation spans being resampled, so latency scales only modestly.
  • SimPO fine-tuning internalizes the BoN gains, giving equal or better citation quality in a single forward pass (77.9 F1) and eliminating BoN's latency overhead.
  • A fully self-supervised pipeline (ContextCite-generated SFT data plus SelfCite) cuts the gap to the supervised LongCite-8B from 8.1 to 2.3 F1 points (65.7 to 71.5 with BoN), suggesting that human or SFT annotation may be replaceable.
  • Iterative SimPO over three rounds keeps improving F1, indicating the reward remains reliable under the model's evolving output distribution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reward measures causal contribution to the model's own generation, not semantic support; on claims where the model relies on spurious or biased context signals, the reward could rank a citation that triggers the response above one a human would judge as evidence. The paper itself notes this contributive/corroborative mismatch is not guaranteed to align.
  • Because the method requires access to token-level output probabilities, it cannot be applied directly to closed-source APIs; a testable extension would be to distill a probability-estimating surrogate or to approximate the reward with sampling-based estimates.
  • The ablation-based reward could serve as a general-purpose attribution quality metric beyond citations—for example, evaluating RAG faithfulness or selecting evidence in retrieval pipelines—since it is annotation-free and model-specific.
  • A concrete stress test: if the reward is used to filter cited sentences generated by a deliberately hallucinating model, does citation F1 still improve without answer correctness collapsing? The paper reports correctness is unchanged, but the test would bound the method's safety.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces SelfCite, a self-supervised method for improving sentence-level citation quality in long-form question answering. A reward is computed from the language model's own probabilities under context ablation: removing the cited sentences should lower the probability of the response statement (necessity), and retaining only the cited sentences should keep it high (sufficiency); the two scores are summed into a single reward (Eq. 1). This reward is used in two ways: best-of-N reranking of citation candidates (Section 2.3) and SimPO preference optimization (Section 2.4). Experiments on LongBench-Cite across five datasets report that applying best-of-N to a SimPO-finetuned LongCite-8B raises average citation F1 from a reproduced baseline of 73.8 to 79.1, a gain of 5.3 points, and that the SimPO model alone reaches 77.9 F1. The paper also includes a fully self-supervised setting where an SFT model is trained on ContextCite-generated citations, ablations of the reward components, length balancing, training-size studies, and a comparison against an NLI-reward baseline.

Significance. If the empirical claims hold, this is a valuable result: it shows that citation quality can be improved without human annotation or proprietary API supervision at the alignment stage, using a simple and interpretable probability-based reward. The paper includes several commendable controls: the NLI-reward SimPO baseline uses the same pipeline with only the reward changed; the ablation in Table 3 fixes the candidate set and varies only the reward; the length-balancing ablation isolates a known shortcut; and the authors explicitly acknowledge the contributive-versus-corroborative mismatch in Section 5. The code is released, which aids reproducibility. The main weakness is that the headline numerical claim is supported by single-run point estimates with an unexplained gap between the reproduced baseline and the published LongCite-8B number, so the central quantitative claim is not yet fully verified.

major comments (3)
  1. [Section 3.4, Table 1] The paper reports no variance or significance information for any of the main F1 numbers. Generation is stochastic (temperature 0.95, top-p 0.7, and temperature 1.2 for best-of-N sampling), and the citation F1 judge is a nondeterministic GPT-4o model. On the dataset sizes typical of LongBench-Cite, a small number of judge disagreements or one unlucky seed could plausibly move average F1 by several points. Please report variance across multiple seeds, a bootstrap over evaluated instances, or at least a judge-sensitivity analysis, and state whether the reported numbers are averages over repeats.
  2. [Table 1, LongCite-8B rows] The reproduced LongCite-8B baseline is 73.8 average F1, while the published LongCite-8B row is 72.0. This 1.8-point gap is never explained. If the reproduction pipeline is systematically more lenient (different GPT-4o version, different judge prompt, different decoding configuration), then the absolute comparison to all published baselines is distorted. Please reconcile the gap by running the official evaluation code, documenting any pipeline differences, and showing that the 5.3-point gain over the reproduction is robust to those differences.
  3. [Section 5 and Section 3.4] The paper's central assumption is that the contributive reward transfers to the corroborative evaluation metric. The authors state this alignment is 'at least partially aligned' and 'not guaranteed,' which is honest, but the empirical evidence for transfer is currently only the five point estimates in Table 1. Without per-dataset error bars or a simple sign test over the five datasets, it is hard to know whether the consistent direction of improvement is statistically reliable or consistent with noise. Please add a per-dataset uncertainty estimate or another explicit transfer test.
minor comments (4)
  1. [Section 5, Related Work] The sentence 'it depends on larger proprietary models two proprietary APIs' is missing a word; it should read 'it depends on two larger proprietary APIs' or similar.
  2. [Footnote 1, Section 3.1] The footnote reports that after deduplication only 4.8 candidates remain per statement on average, but the text still refers to N=10 candidates. Please clarify whether the reward selection operates over the deduplicated set and whether the effective candidate count varies across statements.
  3. [Table 3, caption] The caption says citation lengths are computed across all LongBench-Cite datasets, but the table has a single 'Citation Length' column next to HotpotQA-specific R/P/F1. Please clarify in the caption that R/P/F1 are for HotpotQA while citation length is the LongBench-Cite average, or split them into separate columns.
  4. [Appendix A] The hyperparameter selection for SimPO is described as choosing a learning rate on a 100-example development set, but no development-set results are shown. Reporting the selected learning rate and the sensitivity of the final F1 to it would help reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the SelfCite reward is a model-probability contrast and is validated against an external GPT-4o benchmark, not against its own inputs.

full rationale

We walked the claimed derivation chain. The reward in Eq. (1), Reward(e_i) = log p_LM(r_i | E_i) - log p_LM(r_i | C without E_i), is a causal contrast over the model's own probabilities; it is not defined in terms of LongBench-Cite's citation recall or precision, nor in terms of the GPT-4o judge. Best-of-N sampling uses this reward only to rerank candidate citation strings, and SimPO is trained on preference pairs created by that reranking, so the optimization signal is self-supervised and separate from the external evaluation. The paper's headline improvement is measured on LongBench-Cite with GPT-4o, which is independent of the reward construction, and the contributive-to-corroborative mismatch is explicitly acknowledged in Section 5 rather than hidden. The only notable author-overlap citation is ContextCite (Cohen-Wang et al., 2024), used as inspiration and as a reference baseline; this is not load-bearing because the main result does not depend on any unverified claim imported from that paper, and no uniqueness theorem or fitted benchmark parameter is invoked. The unexplained 1.8-F1 gap between the reproduced LongCite-8B baseline (73.8) and the published value (72.0), and the absence of variance estimates, are legitimate empirical-verification concerns but are not circularity. We therefore find no step in which a prediction reduces by construction to its input.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The approach is methodologically self-contained but rests on domain assumptions about LM probability calibration and the transfer from contributive to corroborative citation quality. No new physical or model-level entities are introduced.

free parameters (4)
  • Lmax citation length cap = 384 tokens
    BoN candidates citing more than 384 tokens are excluded unless a single sentence; ablations in Table 3 show 256 vs 512 changes F1 by about 1 point.
  • BoN candidate count N = 10
    Number of resampled citation candidates per statement; deduplicates to 4.8 on average (Section 3.1).
  • ContextCite SFT thresholds = t=1.5, p=0.7, k=4
    Used to convert attribution scores into citations for the fully self-supervised SFT data; set without reported ablations (Appendix B).
  • SimPO learning rate = selected from 1e-7 to 7e-7
    Picked on a 100-example development set (Appendix A).
assumptions (4)
  • domain assumption Next-token probability changes under context ablation are a valid proxy for evidence necessity and sufficiency.
    The reward in Eq. (1) rests on this behavioral assumption about the LM's output distribution (Section 2.2).
  • domain assumption Contributive attribution is at least partially aligned with corroborative attribution as measured by the benchmark.
    Stated as not guaranteed in Section 5, yet the reported gains depend on this transfer.
  • domain assumption The LongBench-Cite GPT-4o-based evaluation is a valid measurement of citation quality.
    All citation F1 numbers come from this external judge; no independent validation is given (Section 3.3).
  • domain assumption The base model can already generate structured citation candidates.
    BoN only reranks citation strings; the main experiments start from LongCite-8B, which was trained on annotated data. Acknowledged as limitation 3 in Section 6.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models." pith.science (2026). https://pith.science/paper/Q6VUNR6W

@misc{pith2026250209604,
  author       = {Pith},
  title        = {Pith review of: SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q6VUNR6W}},
  note         = {Machine review of arXiv:2502.09604}
}
read the original abstract

We introduce SelfCite, a novel self-supervised approach that aligns LLMs to generate high-quality, fine-grained, sentence-level citations for the statements in their generated responses. Instead of only relying on costly and labor-intensive annotations, SelfCite leverages a reward signal provided by the LLM itself through context ablation: If a citation is necessary, removing the cited text from the context should prevent the same response; if sufficient, retaining the cited text alone should preserve the same response. This reward can guide the inference-time best-of-N sampling strategy to improve citation quality significantly, as well as be used in preference optimization to directly fine-tune the models for generating better citations. The effectiveness of SelfCite is demonstrated by increasing citation F1 up to 5.3 points on the LongBench-Cite benchmark across five long-form question answering tasks. The source code is available at https://github.com/facebookresearch/SelfCite

Figures

Figures reproduced from arXiv: 2502.09604 by the authors.

Figure 1
Figure 1. The SelfCite framework calculates rewards based on two metrics: necessity score (probability drop) and sufficiency score (probability hold). First, the full context is used to generate a response. Then, the framework evaluates the probability of generating the same response after (1) removing the cited sentences from the context and (2) using only the cited sentences in the context. The probability drop and hold are… view at source ↗
Figure 2
Figure 2. Iteratively applying SimPO for three iterations. Specifically, after fine-tuning with SimPO, we generate a new dataset via BoN, which is also 2K in size but not over￾lapped with previous iterations. We continue training the model and repeat the process for three rounds. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. How Context Attribution Handles What the Model Already Knows

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Context attribution methods cannot disentangle in-context from in-weight knowledge and assign unfaithful scores under overlap; new metrics and WMDP-Cyber++ quantify the failure.

  2. On Improving Faithfulness of Podcasts from Documents

    cs.CL 2026-07 conditional novelty 6.0 of 10

    AI-generated podcasts often add unsupported claims; a turn-level detector plus rewrite pass improves measured faithfulness across five models and in- and out-of-domain documents.

  3. MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answering

    cs.CL 2025-05 conditional novelty 6.0 of 10

    LLM relevance judgments align with physician trainees on only about 45 to 66 percent of sentences, and pruning contexts to physician-labeled relevant sentences improves LLM and trainee accuracy.

  4. TRUE: A Trustworthy Unified Explanation Framework for Large Language Model Reasoning

    cs.LG 2026-02 reject novelty 5.0 of 10

    TRUE checks whether LLM reasoning traces are self-sufficient by executing them blind, maps neighboring reasoning paths into a DAG, and ranks recurring failure modes by Shapley values.

  5. LCDS: A Logic-Controlled Discharge Summary Generation System Supporting Source Attribution and Expert Review

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A logic-controlled pipeline with source mapping and sentence-level attribution generates discharge summaries that score higher than a GPT-4o chain-of-thought baseline in this study.

Reference graph

Works this paper leans on

45 extracted references · 18 canonical work pages · cited by 5 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Anthropic: Introducing claude 3.5 sonnet, 2024

    Anthropic. Anthropic: Introducing claude 3.5 sonnet, 2024. URL https://www.anthropic.com/news/claude-3-5-sonnet

    2024

  3. [3]

    Training a helpful and harmless assistant with reinforcement learning from human feedback

    Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

    arXiv 2022

  4. [4]

    Longbench: A bilingual, multitask benchmark for long context understanding

    Bai, Y., Lv, X., Zhang, J., Lyu, H., Tang, J., Huang, Z., Du, Z., Liu, X., Zeng, A., Hou, L., et al. Longbench: A bilingual, multitask benchmark for long context understanding. arXiv preprint arXiv:2308.14508, 2023

    arXiv 2023

  5. [5]

    Longalign: A recipe for long context alignment of large language models

    Bai, Y., Lv, X., Zhang, J., He, Y., Qi, J., Hou, L., Tang, J., Dong, Y., and Li, J. Longalign: A recipe for long context alignment of large language models. arXiv preprint arXiv:2401.18058, 2024

    arXiv 2024

  6. [6]

    Nltk: the natural language toolkit

    Bird, S. Nltk: the natural language toolkit. In Proceedings of the COLING/ACL 2006 Interactive Presentation Sessions, pp.\ 69--72, 2006

    work page 2006

  7. [7]

    Lookback lens: Detecting and mitigating contextual hallucinations in large language models using only attention maps

    Chuang, Y.-S., Qiu, L., Hsieh, C.-Y., Krishna, R., Kim, Y., and Glass, J. Lookback lens: Detecting and mitigating contextual hallucinations in large language models using only attention maps. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 1419--1436, 2024

    work page 2024

  8. [8]

    Contextcite: Attributing model generation to context

    Cohen-Wang, B., Shah, H., Georgiev, K., and Madry, A. Contextcite: Attributing model generation to context. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    work page 2024

Show all 45 references
  1. [9]

    The llama 3 herd of models

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  2. [10]

    T., Fan, Y., Zhao, V

    Gao, L., Dai, Z., Pasupat, P., Chen, A., Chaganty, A. T., Fan, Y., Zhao, V. Y., Lao, N., Lee, H., Juan, D.-C., et al. Rarr: Researching and revising what language models say, using language models. arXiv preprint arXiv:2210.08726, 2022

    arXiv

  3. [11]

    Scaling laws for reward model overoptimization

    Gao, L., Schulman, J., and Hilton, J. Scaling laws for reward model overoptimization. In International Conference on Machine Learning, pp.\ 10835--10866. PMLR, 2023 a

    2023

  4. [12]

    Enabling large language models to generate text with citations

    Gao, T., Yen, H., Yu, J., and Chen, D. Enabling large language models to generate text with citations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 6465--6488, 2023 b

    2023

  5. [13]

    GLM, T., Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Rojas, D., Feng, G., Zhao, H., Lai, H., Yu, H., Wang, H., Sun, J., Zhang, J., Cheng, J., Gui, J., Tang, J., Zhang, J., Li, J., Zhao, L., Wu, L., Zhong, L., Liu, M., Huang, M., Zhang, P., Zheng, Q., Lu, R., Duan, S., Zhan...

    2024

  6. [14]

    Dureader: a chinese machine reading comprehension dataset from real-world applications

    He, W., Liu, K., Liu, J., Lyu, Y., Zhao, S., Xiao, X., Liu, Y., Wang, Y., Wu, H., She, Q., et al. Dureader: a chinese machine reading comprehension dataset from real-world applications. In Proceedings of the Workshop on Machine Reading for Question Answering, pp.\ 37--46, 2018

    2018

  7. [15]

    The curious case of neural text degeneration

    Holtzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y. The curious case of neural text degeneration. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=rygGQyrFvH

    2020

  8. [16]

    Liger kernel: Efficient triton kernels for llm training

    Hsu, P.-L., Dai, Y., Kothapalli, V., Song, Q., Tang, S., Zhu, S., Shimizu, S., Sahni, S., Ning, H., and Chen, Y. Liger kernel: Efficient triton kernels for llm training. arXiv preprint arXiv:2410.10989, 2024. URL https://arxiv.org/abs/2410.10989

    2024 arXiv

  9. [17]

    Training language models to generate text with citations via fine-grained rewards

    Huang, C., Wu, Z., Hu, Y., and Wang, W. Training language models to generate text with citations via fine-grained rewards. arXiv preprint arXiv:2402.04315, 2024 a

    2024 arXiv

  10. [18]

    Efficient attentions for long document summarization

    Huang, L., Cao, S., Parulian, N., Ji, H., and Wang, L. Efficient attentions for long document summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.\ 1419--1436, 2021

    2021

  11. [19]

    Advancing large language model attribution through self-improving

    Huang, L., Feng, X., Ma, W., Zhao, L., Fan, Y., Zhong, W., Xu, D., Yang, Q., Liu, H., and Qin, B. Advancing large language model attribution through self-improving. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 3822--3836, 2024 b

    2024

  12. [20]

    J., Madotto, A., and Fung, P

    Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., and Fung, P. Survey of hallucination in natural language generation. ACM Computing Surveys, 55 0 (12): 0 1--38, 2023

    2023

  13. [21]

    Aligning large language models through synthetic feedback

    Kim, S., Bae, S., Shin, J., Kang, S., Kwak, D., Yoo, K., and Seo, M. Aligning large language models through synthetic feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 13677--13700, 2023

    2023

  14. [22]

    Rationalizing neural predictions

    Lei, T., Barzilay, R., and Jaakkola, T. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp.\ 107--117, 2016

    2016

  15. [23]

    Let's verify step by step

    Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=v8L0pN6EOi

    2024

  16. [24]

    Sim PO : Simple preference optimization with a reference-free reward

    Meng, Y., Xia, M., and Chen, D. Sim PO : Simple preference optimization with a reference-free reward. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=3Tzcot1LKb

    2024

  17. [25]

    Teaching language models to support answers with verified quotes

    Menick, J., Trebacz, M., Mikulik, V., Aslanides, J., Song, F., Chadwick, M., Glaese, M., Young, S., Campbell-Gillingham, L., Irving, G., et al. Teaching language models to support answers with verified quotes. arXiv preprint arXiv:2203.11147, 2022

    2022 arXiv

  18. [26]

    Mistral large, 2024

    Mistral. Mistral large, 2024. URL https://mistral.ai/news/mistral-large/

    2024

  19. [27]

    Controlled decoding from language models

    Mudgal, S., Lee, J., Ganapathy, H., Li, Y., Wang, T., Huang, Y., Chen, Z., Cheng, H.-T., Collins, M., Strohman, T., et al. Controlled decoding from language models. In International Conference on Machine Learning, pp.\ 36486--36503. PMLR, 2024

    2024

  20. [28]

    Webgpt: Browser-assisted question-answering with human feedback

    Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021

    2021 arXiv

  21. [29]

    Introducing chatgpt, November 2022

    OpenAI. Introducing chatgpt, November 2022. URL https://openai.com/blog/chatgpt

    2022

  22. [30]

    Gpt-4 technical report, 2023

    OpenAI. Gpt-4 technical report, 2023. URL https://cdn.openai.com/papers/gpt-4.pdf

    2023

  23. [31]

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., L. Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R. Training language models to f...

    2022 arXiv

  24. [32]

    Y., Yuan, W., Cho, K., He, H., Sukhbaatar, S., and Weston, J

    Pang, R. Y., Yuan, W., Cho, K., He, H., Sukhbaatar, S., and Weston, J. Iterative reasoning preference optimization. arXiv preprint arXiv:2404.19733, 2024

    2024 arXiv

  25. [33]

    Phukan, A., Somasundaram, S., Saxena, A., Goswami, K., and Srinivasan, B. V. Peering into the mind of language models: An approach for attribution in contextual question answering. arXiv preprint arXiv:2405.17980, 2024

    2024 arXiv

  26. [34]

    Model internals-based answer attribution for trustworthy retrieval-augmented generation

    Qi, J., Sarti, G., Fern \'a ndez, R., and Bisazza, A. Model internals-based answer attribution for trustworthy retrieval-augmented generation. arXiv preprint arXiv:2406.13663, 2024

    2024 arXiv

  27. [35]

    D., Ermon, S., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  28. [36]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. In arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  29. [37]

    Trusting your evidence: Hallucinate less with context-aware decoding

    Shi, W., Han, X., Lewis, M., Tsvetkov, Y., Zettlemoyer, L., and Yih, W.-t. Trusting your evidence: Hallucinate less with context-aware decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language...

    2024

  30. [38]

    Attribute first, then generate: Locally-attributable grounded text generation

    Slobodkin, A., Hirsch, E., Cattan, A., Schuster, T., and Dagan, I. Attribute first, then generate: Locally-attributable grounded text generation. arXiv preprint arXiv:2403.17104, 2024

    2024 arXiv

  31. [39]

    L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A

    Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. Transformers: St...

    2020

  32. [40]

    H., Meister, N., Winston, C., and Guestrin, C

    Worledge, T., Shen, J. H., Meister, N., Winston, C., and Guestrin, C. Unifying corroborative and contributive attributions in large language models. arXiv preprint arXiv:2311.12233, 2023

    2023 arXiv

  33. [41]

    W., Salakhutdinov, R., and Manning, C

    Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., and Manning, C. D. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600, 2018

    2018 arXiv

  34. [42]

    Alma: Alignment with minimal annotation

    Yasunaga, M., Shamis, L., Zhou, C., Cohen, A., Weston, J., Zettlemoyer, L., and Ghazvininejad, M. Alma: Alignment with minimal annotation. arXiv preprint arXiv:2412.04305, 2024

    2024 arXiv

  35. [43]

    Y., Cho, K., Li, X., Sukhbaatar, S., Xu, J., and Weston, J

    Yuan, W., Pang, R. Y., Cho, K., Li, X., Sukhbaatar, S., Xu, J., and Weston, J. E. Self-rewarding language models. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=0NphYCmgua

    2024

  36. [44]

    Longcite: Enabling llms to generate fine-grained citations in long-context qa

    Zhang, J., Bai, Y., Lv, X., Gu, W., Liu, D., Zou, M., Cao, S., Hou, L., Dong, Y., Feng, L., et al. Longcite: Enabling llms to generate fine-grained citations in long-context qa. arXiv preprint arXiv:2409.02897, 2024

    2024 arXiv

  37. [45]

    LIMA : Less is more for alignment

    Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., YU, L., Zhang, S., Ghosh, G., Lewis, M., Zettlemoyer, L., and Levy, O. LIMA : Less is more for alignment. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Adva...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.