Pith. sign in

REVIEW 10 cited by

Inspecting and Editing Knowledge Representations in Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2304.00740 v3 pith:4CQZHK6Y submitted 2023-04-03 cs.CL

Evan Hernandez , Belinda Z. Li , Jacob Andreas This is my paper · ORCID
classification cs.CL
keywords factsknowledgeremediencodingsfactlanguagerepresentationrepresentations
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Neural language models (LMs) represent facts about the world described by text. Sometimes these facts derive from training data (in most LMs, a representation of the word "banana" encodes the fact that bananas are fruits). Sometimes facts derive from input text itself (a representation of the sentence "I poured out the bottle" encodes the fact that the bottle became empty). We describe REMEDI, a method for learning to map statements in natural language to fact encodings in an LM's internal representation system. REMEDI encodings can be used as knowledge editors: when added to LM hidden representations, they modify downstream generation to be consistent with new facts. REMEDI encodings may also be used as probes: when compared to LM representations, they reveal which properties LMs already attribute to mentioned entities, in some cases making it possible to predict when LMs will generate outputs that conflict with background knowledge or input text. REMEDI thus links work on probing, prompting, and LM editing, and offers steps toward general tools for fine-grained inspection and control of knowledge in LMs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 16 citations worldwide. Full citation record

  1. Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias

    cs.LG 2026-07 conditional novelty 6.5 of 10

    LLM-as-judge scoring biases concentrate in low-dimensional, type-specific activation subspaces that support bidirectional causal steering and cross-domain failure prediction.

  2. Mind the Quote: Enabling Quotation-Aware Dialogue in LLMs via Plug-and-Play Modules

    cs.AI 2025-05 conditional novelty 6.5 of 10

    QuAda, a trainable attention adapter using under 2.8% extra parameters, gives instruction-tuned LLMs strong performance on five quotation-aware dialogue tasks.

  3. Targeted Recovery of Weight-Space Mechanisms From Neural Networks

    cs.LG 2026-06 conditional novelty 6.0 of 10

    A targeted decomposition method recovers the weight-space mechanisms behind specific inputs at low FLOPs, enabling focused ablation and rewiring of a 12-block transformer.

  4. Emergent Response Planning in LLMs

    cs.CL 2025-02 conditional novelty 6.0 of 10

    Hidden representations of LLM prompts encode global attributes of the upcoming response, and simple probes can predict length, content choices, and answer confidence before generation begins.

  5. Probe-Free Low-Rank Activation Intervention

    cs.LG 2025-02 conditional novelty 6.0 of 10

    FLORAIN is a probe-free, single-layer activation intervention that improves LLM truthfulness by projecting hidden states toward an ellipsoidal region of desirable answers.

  6. REFLEX: Self-Refining Explainable Fact-Checking via Verdict-Anchored Style Control

    cs.CL 2025-11 unverdicted novelty 5.0 of 10

    REFLEX improves explainable fact-checking by using verdict-anchored style control and self-disagreement signals to disentangle fact from style in LLM outputs, achieving SOTA results with minimal self-refined samples.

  7. Balancing Stylization and Truth via Disentangled Representation Steering

    cs.CL 2025-08 reject novelty 5.0 of 10

    StyliTruth separates style and truth directions in the activations of selected attention heads, then steers each token along the disentangled subspaces to preserve truthfulness during stylization.

  8. Activation Reward Models for Few-Shot Model Alignment

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Mean attention-head activations from a few labeled examples, injected into selected heads, turn a frozen vision-language model into a few-shot reward model that beats prompting and scoring baselines and a new reward-h...

  9. Ask Good Questions for Large Language Models

    cs.CL 2025-08 unverdicted novelty 4.0 of 10

    The AGQ framework combines a concept-enhanced item response theory model with LLMs to generate guiding questions that adapt to a user's estimated knowledge gaps.

  10. Task-driven Layerwise Additive Activation Intervention

    cs.CL 2025-02 conditional novelty 4.0 of 10

    A lasso and group-lasso regularized additive intervention on last-token activations steers LLMs with 10 demonstrations and supports task-vector composition.

Pith tools