Abstract Counterfactuals for Language Model Agents

David Watson; Edoardo Pona; Milad Kazemi; Nicola Paoletti; Yali Du

REVIEW 3 major objections 5 minor 43 references

Abstract Counterfactuals for Language Model Agents

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that counterfactual inference for language-model agents should be performed over high-level semantic abstractions of actions, not over the tokens the model outputs, and presents experiments showing this yields far more…

desk verdict New abstraction-level counterfactual idea for LM agents; the consistency proof is misstated and the Machiavelli experiment needs explanation, but the core deserves a serious referee. read the letter →

arxiv 2506.02946 v1 pith:3BZYVR4H submitted 2025-06-03 cs.LG

Edoardo Pona , Milad Kazemi , Yali Du , David Watson , Nicola Paoletti This is my paper

classification cs.LG

keywords abstractcounterfactualslanguagemodelagentsstructuralcausalmodelscounterfactualinferencesemanticabstractiontoken-leveltextgenerationchoice-basedenvironments

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Counterfactual questions about language-model agents—"what would this agent have done in a different situation?"—are hard because agents act by producing arbitrary strings, and the same token can mean different things in different contexts. This paper proposes asking the question at the level of an abstraction $Y$ that captures the high-level meaning of the action (cautious or reckless, profession, emotion), and then mapping the counterfactual abstraction back to concrete actions. The experiments claim that this abstract procedure keeps the semantic content of the counterfactual aligned with the factual action far more often than token-level methods do, while still delivering the intervention's intended effect, for example changing gender while preserving profession. If the claim holds, ACF gives a black-box method for meaningful "what if" analysis of LM agents without inverting their token-generation internals.

What carries the argument

The load-bearing object is the abstraction variable $Y_t = f_Y(A_t, S_t, U^Y_t)$, a context-dependent semantic summary of the agent's action, together with the mapping-back formula (3) that reweights candidate actions by $P_Y(y' \mid s', a')$, i.e. by how likely each action is to produce the counterfactual abstraction. The argument is carried by two structural choices: abduction is done on the combined mechanism $f_Y \circ f_A$ while conditioning only on the observed abstraction value, and the final action is sampled from the posterior $P_{A'}$ that marginalizes over the counterfactual abstraction. Proposition 1 guarantees that averaging these individual counterfactual distributions over the prior noise of $U_Y$ recovers the interventional distribution, which anchors the method to standard causal semantics.

What would settle it

Train the abstraction on labels that are known not to reflect the agent's action semantics, or on random labels, and rerun the same comparisons: if the consistency advantage over token-level methods survives, the claimed mechanism—semantic fidelity—is not what drives it.

Watch

Extended reading notes

Core claim

The paper's central claim is that a counterfactual action for an LM agent can and should be derived through the action's semantic abstraction, not through the token-sampling mechanism. After observing action $a$ in state $s$ and its abstraction $y = f_Y(a,s)$, the method abducts the noise $U'_Y = U_Y \mid s, y$, predicts a counterfactual abstraction distribution in the new state $s'$, and maps back to actions with the Bayes reweighting $P_{A'}(a' \mid y', s') = P_Y(y' \mid s', a')\, P_A(a' \mid s') / P_Y(y' \mid s')$. The observed token $a$ is deliberately omitted from the conditioning, so the procedure cannot merely inflate the probability of repeating the same token; it must instead favour actions that express the same meaning in the new context. This is what the paper means by preserving high-level action meaning across counterfactual contexts.

Load-bearing premise

ACF assumes that the abstraction variable $Y$ faithfully and stably captures the high-level meaning of the agent's action across contexts, and that the distribution $P_Y$ used in the procedure (certified by annotations, a classifier, or an LLM) is a valid stand-in for the agent's own abstraction mechanism $f_Y$.

Editorial extensions

If this is right

In choice-based games, ACF remains well defined when the factual and counterfactual action menus differ in size or share no token labels, a case where token-level Gumbel counterfactuals require padding or truncating noise.
In open-text generation, ACF can change the surface text while holding the high-level content fixed; the reported Abstraction Change Rate drops from 0.38–0.67 with token-level methods to 0.02–0.41 with ACF across the two text tasks.
The method needs only black-box samples from the language model; the token-level mechanism $f_A$ is never inverted, so no internal logits or weights are required.
Counterfactual outputs stay coherent with each other: ACF wins the semantic-tightness comparison against token-level counterfactuals in all six reported configurations.
Abstractions can be supplied in three ways—expert annotations, supervised classifiers, or unsupervised LLM-discovered topic clusters—so the framework is usable in settings without a predefined action space.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implication the paper leaves implicit: anchoring counterfactuals on $Y$ changes the unit of explanation from words to properties, so questions about blame or responsibility become about intents and features rather than about particular strings; downstream causal-explanations discussions would inherit that shift.
The paper never verifies that the external classifier's $P_Y$ matches the agent's own abstraction mechanism $f_Y$; a natural next experiment is to vary the abstraction scheme while fixing everything else and check whether ACF's advantage tracks the abstraction's fidelity to the agent's semantics.
Because abduction conditions on the abstraction value only, ACF predicts the same counterfactual distribution for any two observed actions that map to the same $y$; this is a testable signature—two actions with identical labels but different token-level identities should nonetheless yield identical ACF counterfactuals.
If $Y$ is degenerate (one class), ACF collapses to the interventional distribution, and if $Y$ becomes as detailed as the action itself it approaches token-level behaviour; charting this spectrum would map precisely where token-level methods start to fail.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea is genuinely new: instead of doing counterfactuals on the token mechanism, condition on a semantic abstraction Y of the action, predict Y' in the counterfactual state, and map back. The three-step procedure is clean and black-box. The paper is also honest about needing to define an abstraction and about compute cost. The comparisons against TLCF on gender steering and token replacement show large differences, and the semantic tightness metric plus the worked gender example are independent evidence that something real is happening.

But the formal guarantee as stated does not cover the algorithm. Proposition 1 averages over the prior of UY, while the abduction step in Eqs. (3)-(4) uses the posterior. The stress-test note is right: that makes the proposition a prior-averaged identity, not a statement about the posterior-conditioned counterfactual. For a deterministic abstraction (MACHIAVELLI), the posterior equals the prior, so the method cannot transmit the observed y; the ACF distribution would be interventional. Yet Figure 3 shows precisely a dependence on the observed abstraction. The implementation must be doing something else (maybe setting Y'=y before mapping back), but it isn't described. That is the paper's most serious soft spot.

Two more concerns, in proportion. The headline metrics ACR and CPIR measure whether the observed abstraction is preserved—which is exactly what the conditioning step forces. Part of the advantage over TLCF is by construction; the ST metric and the qualitative examples carry more weight than the tables. And the abstraction classifiers (DistilBERT, LLM) are treated as if they were the agent's own f_Y; the paper never checks this fidelity assumption. Minor: no code, no sampling details, no confidence intervals on the main numbers.

Who benefits: anyone working on causal explanations or interpretability of LM agents. The idea is worth airing, and the flaws are addressable. A serious referee should see it, but with the expectation of heavy revision: fix or rescope the proposition, explain or correct the Machiavelli experiment, and add code and error bars.

Referee Report

3 major / 5 minor

Summary. The paper introduces Abstract Counterfactuals (ACF), a framework for counterfactual inference on language-model agents. Instead of performing abduction on the token-level Gumbel mechanism, ACF introduces an abstraction variable Y that represents the high-level meaning of an action in a given state, performs abduction on Y's exogenous noise, predicts a counterfactual abstraction Y' in the counterfactual state, and then maps Y' back to an action distribution via Bayes' rule (Eqs. 3-4). The authors claim Proposition 1 as a consistency property: averaging the ACF distribution over the prior of the exogenous noise recovers the interventional distribution. The method is evaluated on three settings: MACHIAVELLI choice-based games with deterministic annotation-based abstractions, Bios biography generation with a gender-steering latent intervention, and GoEmotions with token replacement interventions. The paper reports that ACF maintains much higher semantic consistency than token-level Gumbel counterfactuals (TLCF), with lower Abstraction Change Rate and higher Counterfactual Probability Increase Rate.

Significance. If the central claim were sound, ACF would be a valuable contribution to causal analysis of LM agents: it is black-box, avoids token-level semantic drift, and provides an interpretable abstraction layer. The empirical results are impressive in magnitude (ACR 0.02-0.12 vs 0.32-0.67), and the unsupervised abstraction pipeline using an auxiliary LLM is a useful methodological addition. The paper also correctly identifies real failure modes of token-level counterfactuals. However, the main theoretical guarantee, Proposition 1, is not about the distribution that the algorithm actually computes, and the deterministic-abstraction experiment in MACHIAVELLI appears to degenerate to the interventional policy. These issues directly affect the claim that the approach 'produces consistent and meaningful counterfactuals'. As a result, the paper as it stands does not yet substantiate its central contribution, though the underlying idea remains promising and potentially fixable.

major comments (3)

[Section 3.1 and Appendix A] The statement of Proposition 1 concerns the expectation of the ACF distribution over U'_Y drawn from the prior P(UY), but the algorithm's Step 1 derives the posterior P(UY | s, y). The proof's first line, Σ_{u'} P(u') ..., confirms that the proposition is about prior-averaged behavior, not about the posterior-conditioned distribution that the method actually computes. Consequently, the proposition does not establish any consistency property for the output of Eqs. (3)-(4). The defect is load-bearing: in the MACHIAVELLI experiment (Sec. 4.1) P_Y is degenerate, so UY has a point-mass posterior equal to the prior; then Step 2 yields Y' distributed as the interventional P(y'|s'), and Eq. (4) collapses to PA'(a'|s') = PA(a'|s'). The reported behavior in Figure 3 (increased probability of actions with the observed annotation 'physical harm:1') therefore cannot be produced by the method as described; the figure is inconsistent with Eqs. (3)-(4) unless an additional step (e.g., setting Y'=y) is introduced. The main formal guarantee is thus unsupported, and the Machiavelli demonstration is not an instance of the stated method.
[Section 3 and Sections 4.1-4.3] The SCM in Sec. 3 posits Yt = fY(At, St, UYt) as a structural node of the agent's data-generating process. In the experiments, however, Y is realized as an external artifact: benchmark annotations in Sec. 4.1, a fine-tuned DistilBERT classifier in Appendix E, and an LLM topic assignment in Appendix F. The paper never provides evidence that these external distributions are faithful stand-ins for the agent's own abstraction mechanism. In particular, the premise that Y is a context-stable semantic proxy (Sec. 3) is asserted but not tested, and the validity of conditioning on Y alone (dropping the observed action a) is not verified. A concrete test would be to compare counterfactual predictions made with the external Y against observed behavior in a held-out counterfactual state, or to measure whether the learned PY is invariant across contexts. Without such validation, the claim that the resulting counterfactuals are 'meaningful' for the agent is not established.
[Section 3.1 and Appendices E-F] The implementation of the abduction step is not specified. The algorithm requires computing or estimating the posterior P(UY | s, y) and the quantities P(y | s, u'_Y) in Step 1, but the paper's descriptions of the abstraction models (a fine-tuned DistilBERT outputting P(y | s, a), an LLM providing logits over topic indices) do not explain how an explicit exogenous noise UY is defined or how the posterior is obtained. Without these details the method is not reproducible, and it is unclear whether the reported experimental results actually instantiate the three-step procedure of Section 3.1.

minor comments (5)

[Appendix A] The proof's notation conflates the exogenous noise of A and Y and drops the conditioning on s throughout; while the algebraic identity is correct, the notation should be cleaned up to make clear that P(y'|u') means P(y' | s', u').
[Section 4.1] The statement that 'we pad the Gumbel noise vector with 0 values when the counterfactual action space is larger than the factual one' is an ad hoc choice for the TLCF baseline; the paper should justify it or at least acknowledge that it is an arbitrary convention affecting the comparison.
[Section 4.2 and 4.3] The ST rows in Tables 1 and 2 report win rates, but the captions do not state this; please clarify how the ST comparison is computed and what the t-statistics correspond to.
[Appendix B] The notation Pcf(y' | a, s') in the ACR definition is not defined; it should be stated that Pcf is the counterfactual distribution over the abstraction given a counterfactual action and state.
[Figure 3] Since the Machiavelli results are explicitly illustrative, the figure should include a note that the probabilities are point estimates from a single scene and lack error bars or variance information.

Circularity Check

1 steps flagged · score 6.0 of 10

Proposition 1's consistency guarantee is a prior-averaged tautology that discards the posterior conditioning the ACF algorithm actually uses; the main formal support for ACF reduces to an interventional identity.

self definitional [Section 3.1 Proposition 1; Appendix A proof]
"Proposition 1 ... E_{U′_Y ∼ P_UY}(P_{A′}(a′ | s′)) = P_A(a′ | s′), where the expectation is taken over P_UY, the prior distribution of UY. ... So, the LHS of the above statement can be rewritten as Σ_{u′} Σ_{y′} P(u′) · P(y′ | u′) P(y′ | a′) · P(a′) / P(y′)."

Step 1 of ACF sets U′_Y = UY | s,y, the posterior, so the ACF distribution P_{A′} depends on the observed abstraction y. The proposition and proof instead take the expectation over the prior P(UY): the proof's first line contains only P(u′) and P(y′|u′), with no conditioning on the observed y. The equality then follows immediately from the identity P(y′) = E_u P(y′|u) and Bayes' rule, and holds for any stochastic map, without using the posterior at all. The stated 'interventional consistency' is therefore equivalent by construction to the definition of the interventional distribution itself; it provides no support for the posterior-conditioned counterfactual distribution that the method outputs.

full rationale

The paper's empirical comparison against the independent TLCF baseline is meaningful and not circular: ACF and TLCF are evaluated on the same generated samples, and the abstraction classifiers (DistilBERT for Bios/GoEmotions, LLM topic assignment) are external artifacts not fitted to the counterfactual outcomes. However, the only formal guarantee claimed for ACF, Proposition 1, is a tautology over the prior: Appendix A's expansion drops the conditioning on y that Step 1 of the algorithm defines, so the theorem reduces to the interventional identity P(y′)=E_uP(y′|u) and says nothing about whether conditioning on y produces consistent counterfactuals. This is a load-bearing self-definitional step because the abstract's claim that ACF 'produces consistent and meaningful counterfactuals' leans on this consistency property. Separately (a correctness issue, not circularity), Section 4.1's Machiavelli illustration is difficult to reconcile with Eqs. (3)-(4): the annotations make P_Y degenerate, so the posterior equals the prior, Eq. (4) is independent of the observed y, and Figure 3's claim that ACF raises the probability of the observed 'physical harm: 1' action is not a derivable consequence of the stated method. These concerns are substantive but the paper still contains independent experimental content, so the overall circularity score is 6 rather than higher.

Assumptions & free parameters 3 free parameters · 5 assumptions · 2 invented entities

The central claim rests on the token-level Gumbel-Max SCM of prior work [7, 26] (standard), on the new assumption that Y with independent noise U_Y is a faithful cross-context model of the agent's semantics (domain), on the sufficiency of conditioning only on y (domain), and on the normative choice that preserving y in the counterfactual is desirable (domain). In addition, the empirical comparison relies on an ad hoc 0-padding of Gumbel noise for the TLCF baseline in MACHIAVELLI, which the paper acknowledges is undefined. The supervised abstraction classifiers (f1 0.85 professions, 0.63 emotions) are fitted models that stand in for the agent's abstraction mechanism; the unsupervised abstraction uses per-instance LLM topic discovery with an ad hoc 'Other' category.

free parameters (3)

Abstraction variable Y (definition and granularity) = profession (28 classes), emotion (28 classes plus neutral), morality annotations (13 binary labels), or per-instance…
The method's behavior and the interpretation of ACR/CPIR depend entirely on this choice; Sec 3.2 concedes the coarseness balance 'relies on expert judgment'.
Supervised abstraction classifiers (DistilBERT) = profession classifier f1 0.85; emotion classifier f1 0.63
These fitted classifiers supply the PY(y | s, a) used in the conditioning and mapping-back steps (Eqs. 3-4), so classifier errors propagate directly into the counterfactual distributions.
Unsupervised abstraction setup (gpt-4o-mini topic discovery, top-20 logits, 'Other' category) = 'Other' category added when valid topic logits are absent
Defines the sample space of Y separately for each instance; the 'Other' bucket is an ad hoc fix for the OpenAI API logits limit (Appendix F).

assumptions (5)

standard math Autoregressive token generation is faithfully represented by the Gumbel-Max SCM (Eq. 1), with i.i.d. Gumbel noise U per token.
Adopted from Chatzi et al. [7] and Ravfogel et al. [26]; the paper relies on it to define the token-level mechanism f_A and the TLCF baseline.
domain assumption The abstraction Y is a structural variable of the agent with its own exogenous noise U_Y, independent of U_A, whose meaning is consistent across contexts.
Sec 3 defines Yt = fY(At, St, UYt) and claims cross-context validity, but in the experiments Y is an external annotation or a fitted classifier, not a verified internal mechanism of the agent.
domain assumption Conditioning only on the observed abstraction y (not the action a) is sufficient for the counterfactual query.
Sec 3.1: 'ACF's abduction step is conditioned only on the abstraction y, not the action a.' This discards all token-level information about the actual action and is never empirically validated.
domain assumption Preserving the observed abstraction in the counterfactual state is the desired outcome (e.g., profession should not change under gender steering).
Sec 4.2 treats the profession shift of TLCF as an 'undesired side effect'; this normative choice defines what counts as a meaningful counterfactual.
ad hoc to paper For the TLCF baseline in MACHIAVELLI, actions absent from the factual action space receive Gumbel noise padded with 0.
Sec 4.1 acknowledges TLCF 'is not well defined when the cardinality of the presented action space varies'; padding with 0 is an arbitrary fix that can bias the baseline downward.

invented entities (2)

Abstraction variable Y as a structural SCM node of the agent
purpose: Carries high-level action semantics across contexts so counterfactual inference can bypass token-level abduction.
Y is new to this paper's SCM; its validity is supported only by the paper's own consistency metrics, which encode preservation of Y, and by illustrative examples. No external falsifiable handle (e.g., a behavioral prediction testable without the paper's pipeline) is provided.
Posterior over exogenous noise U_Y used in the abduction step
purpose: Formalizes conditioning on the observed abstraction y without conditioning on the action.
The paper never specifies how this posterior is computed or sampled. In the deterministic-annotation setting (MACHIAVELLI) no informative posterior exists, and the reported Figure 3 effect cannot arise from the described equations; an unstated operationalization is required.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Abstract Counterfactuals for Language Model Agents." pith.science (2026). https://pith.science/paper/3BZYVR4H

@misc{pith2026250602946,
  author       = {Pith},
  title        = {Pith review of: Abstract Counterfactuals for Language Model Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3BZYVR4H}},
  note         = {Machine review of arXiv:2506.02946}
}

read the original abstract

Counterfactual inference is a powerful tool for analysing and evaluating autonomous agents, but its application to language model (LM) agents remains challenging. Existing work on counterfactuals in LMs has primarily focused on token-level counterfactuals, which are often inadequate for LM agents due to their open-ended action spaces. Unlike traditional agents with fixed, clearly defined action spaces, the actions of LM agents are often implicit in the strings they output, making their action spaces difficult to define and interpret. Furthermore, the meanings of individual tokens can shift depending on the context, adding complexity to token-level reasoning and sometimes leading to biased or meaningless counterfactuals. We introduce \emph{Abstract Counterfactuals}, a framework that emphasises high-level characteristics of actions and interactions within an environment, enabling counterfactual reasoning tailored to user-relevant features. Our experiments demonstrate that the approach produces consistent and meaningful counterfactuals while minimising the undesired side effects of token-level methods. We conduct experiments on text-based games and counterfactual text generation, while considering both token-level and latent-space interventions.

Figures

Figures reproduced from arXiv: 2506.02946 by the authors.

**Figure 2.** SCM of an LM agent with abstraction variable Y Supervised and unsupervised abstractions. The ACF method leaves us the choice of how to construct this abstraction. Where the user has ‘expert knowledge’ about the domain or an opinionated view on what features of the samples should be considered meaningful, we can use a supervised abstraction. That is, either using annotations directly (as we do in Section 4.1), or us… view at source ↗

**Figure 3.** MACHIAVELLI case study. Action distributions for factual (left) and counterfactual [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Scatterplots showing the semantic tightness of counterfactual samples generated with [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

**Figure 6.** Figure 6: Scenes from the ‘Aegis Project Files 180’ game, abstraction ‘manipulation’. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 7.** Figure 7: Scenes from the ‘Aegis Project Files 180’ game, abstraction ‘physical harm’. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

**Figure 8.** Figure 8: Scenes from the ‘Alexandria’ game, abstraction ‘killing’. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 18 canonical work pages

[1]

Challenges and Opportunities in Text Generation Explainability

Kenza Amara, Rita Sevastjanova, and Mennatallah El-Assady. Challenges and Opportunities in Text Generation Explainability, May 2024. URL http://arxiv.org/abs/2405.08468. arXiv:2405.08468 [cs]

work page Pith review arXiv 2024
[2]

Constructing Effective Personalized Policies Using Counterfactual Inference from Biased Data Sets with Many Features

Onur Atan, William R. Zame, Qiaojun Feng, and Mihaela van der Schaar. Constructing Effective Personalized Policies Using Counterfactual Inference from Biased Data Sets with Many Features, July 2018. URL http://arxiv.org/abs/1612.08082. arXiv:1612.08082 [stat]

work page Pith review arXiv 2018
[3]

Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)

Leo Breiman. Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author). Statistical Science, 16(3):199–231, 08 2001. doi: 10.1214/ss/1009213726. URL http://dx.doi.org/10.1214/ss/1009213726

arXiv 2001
[4]

RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control, July 2023

Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnik...

arXiv 2023
[5]

Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search

Lars Buesing, Theophane Weber, Yori Zwols, Sebastien Racaniere, Arthur Guez, Jean-Baptiste Lespiau, and Nicolas Heess. Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search. arXiv preprint arXiv:1811.06272, 2018

arXiv 2018
[6]

Ruth M. J. Byrne. Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pages 6276–6282, Macao, China, August 2019. International Joint Conferences on Artificial Intelligence Organization. ISBN 978-0-9992411-4-1. doi: 10.24...

work page 2019
[7]

Counterfactual Token Generation in Large Language Models, September 2024

Ivi Chatzi, Nina Corvelo Benz, Eleni Straitouri, Stratis Tsirtsis, and Manuel Gomez-Rodriguez. Counterfactual Token Generation in Large Language Models, September 2024. URL http: //arxiv.org/abs/2409.17027. arXiv:2409.17027 [cs]

arXiv 2024
[8]

Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting, January 2019

Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexan- dra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting, January 2019. URL http://arxiv.org/abs/1901.09451. arXiv:1901.09451

arXiv 2019

Show all 43 references

[9]

GoEmotions: A Dataset of Fine-Grained Emotions, June 2020

Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. GoEmotions: A Dataset of Fine-Grained Emotions, June 2020. URL http: //arxiv.org/abs/2005.00547. arXiv:2005.00547 [cs]

2020 arXiv
[10]

Detecting Hallucinations in Large Language Models using Semantic Entropy

Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting Hallucinations in Large Language Models using Semantic Entropy. Nature, 630(8017):625–630, June 2024. ISSN 1476-4687. doi: 10.1038/s41586-024-07421-0. URL https://www.nature.com/ articles/s41586-024-07421...

2024 doi
[11]

The llama 3 Herd of Models

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 Herd of Models. arXiv preprint arXiv:2407.21783, 2024

2024 arXiv
[12]

Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkin- son, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Ja...

2024 arXiv
[13]

Chawla, Olaf Wiest, and Xiangliang Zhang

Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V . Chawla, Olaf Wiest, and Xiangliang Zhang. Large Language Model based Multi-Agents: A Survey of Progress and Challenges, April 2024. URL http://arxiv.org/abs/2402.01680. arXiv:2402.01680 [cs]

2024 arXiv
[14]

Joseph Y . Halpern. Cause, Responsibility, and Blame: A Structural-Model Approach, December
[15]

Causes and Explanations: A Structural-Model Approach

Joseph Y Halpern and Judea Pearl. Causes and Explanations: A Structural-Model Approach. Part II: Explanations. Br. J. Philos. Sci., 56(4):889–911, 2005

2005
[16]

Coun- terfactual Influence in Markov Decision Processes, February 2024

Milad Kazemi, Jessica Lally, Ekaterina Tishchenko, Hana Chockler, and Nicola Paoletti. Coun- terfactual Influence in Markov Decision Processes, February 2024. URL http://arxiv.org/ abs/2402.08514. arXiv:2402.08514

2024 arXiv
[17]

Jinqi Lai, Wensheng Gan, Jiayang Wu, Zhenlian Qi, and Philip S. Yu. Large Language Models in Law: A Survey. 2023. doi: 10.48550/ARXIV .2312.03718. URL https://arxiv.org/ abs/2312.03718. Publisher: arXiv Version Number: 1

2023 doi arXiv
[18]

A* sampling.Advances in neural information processing systems, 27, 2014

Chris J Maddison, Daniel Tarlow, and Tom Minka. A* sampling.Advances in neural information processing systems, 27, 2014

2014
[19]

Explainable Reinforcement Learning Through a Causal Lens, November 2019

Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere. Explainable Reinforcement Learning Through a Causal Lens, November 2019. URL http://arxiv.org/abs/1905. 10958. arXiv:1905.10958 [cs]

2019 arXiv
[20]

Miao, Eduardo Rodriguez Almaraz, Madhumita Sushil, Atul J

Nikita Mehandru, Brenda Y . Miao, Eduardo Rodriguez Almaraz, Madhumita Sushil, Atul J. Butte, and Ahmed Alaa. Evaluating Large Language Models as Agents in the Clinic.npj Digital Medicine, 7(1):84, April 2024. ISSN 2398-6352. doi: 10.1038/s41746-024-01083-y. URL https://www.na...

2024 doi
[21]

Biases in Large Language Models: Origins, Inventory, and Discussion

Roberto Navigli, Simone Conia, and Björn Ross. Biases in Large Language Models: Origins, Inventory, and Discussion. J. Data and Information Quality, 15(2):10:1–10:21, June 2023. ISSN 1936-1955. doi: 10.1145/3597307. URL https://dl.acm.org/doi/10.1145/3597307

2023 doi
[22]

Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models

Michael Oberst and David Sontag. Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models. In International Conference on Machine Learning, pages 4881–4890. PMLR, 2019

2019
[23]

Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the Machiavelli Benchmark

Alexander Pan, Jun Shern Chan, Andy Zou, Nathaniel Li, Steven Basart, Thomas Woodside, Jonathan Ng, Hanlin Zhang, Scott Emmons, and Dan Hendrycks. Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the Machiavelli Benchmark. ICML, 2023

2023
[24]

Causality

Judea Pearl. Causality. Cambridge University Press, New York, 2009

2009
[25]

Language Models are Unsupervised Multitask Learners

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language Models are Unsupervised Multitask Learners. OpenAI blog, 1(8):9, 2019

2019
[26]

Gumbel Counter- factual Generation From Language Models

Shauli Ravfogel, Anej Svete, Vésteinn Snæbjarnarson, and Ryan Cotterell. Gumbel Counter- factual Generation From Language Models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=TUC0ZT2zIQ

2025
[27]

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Nils Reimers and Iryna Gurevych. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL https: //arxiv.org/abs/1908.10084. 11

2019 arXiv
[28]

Escalation Risks from Language Models in Military and Diplomatic Decision-Making

Juan-Pablo Rivera, Gabriel Mukobi, Anka Reuel, Max Lamparth, Chandler Smith, and Jacquelyn Schneider. Escalation Risks from Language Models in Military and Diplomatic Decision-Making. In The 2024 ACM Conference on Fairness, Accountability, and Trans- parency, pages 836–898, Ri...

2024
[29]

DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter, 2020

Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter, 2020. URL https://arxiv.org/ abs/1910.01108

2020 arXiv
[30]

Representation Surgery: Theory and Practice of Affine Steering, July

Shashwat Singh, Shauli Ravfogel, Jonathan Herzig, Roee Aharoni, Ryan Cotterell, and Pon- nurangam Kumaraguru. Representation Surgery: Theory and Practice of Affine Steering, July
[31]

A Short Survey of Viewing Large Language Models in Legal Aspect

Zhongxiang Sun. A Short Survey of Viewing Large Language Models in Legal Aspect. 2023. doi: 10.48550/ARXIV .2303.09136. URLhttps://arxiv.org/abs/2303.09136. Publisher: arXiv Version Number: 1

2023 doi arXiv
[32]

MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning, June 2024

Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Ar- man Cohan, and Mark Gerstein. MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning, June 2024. URL http://arxiv.org/abs/2311.10537. arXiv:2311.10537 [cs]

2024 arXiv
[33]

Probabilities of Causation: Bounds and Identification

Jin Tian and Judea Pearl. Probabilities of Causation: Bounds and Identification. Ann. Math. Artif. Intell., 28(1-4):287–313, 2000

2000
[34]

Generative Agent-Based Modeling with Actions Grounded in Physical, Social, or Digital Space using Concordia

Alexander Sasha Vezhnevets, John P Agapiou, Avia Aharon, Ron Ziv, Jayd Matyas, Edgar A Duéñez-Guzmán, William A Cunningham, Simon Osindero, Danny Karmon, and Joel Z Leibo. Generative Agent-Based Modeling with Actions Grounded in Physical, Social, or Digital Space using Concord...

2023 arXiv
[35]

V oyager: An Open-Ended Embodied Agent with Large Language Models, October 2023

Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. V oyager: An Open-Ended Embodied Agent with Large Language Models, October 2023. URL http://arxiv.org/abs/2305.16291. arXiv:2305.16291 [cs]

2023 arXiv
[36]

A Survey on Large Language Model based Autonomous Agents

Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen. A Survey on Large Language Model based Autonomous Agents. Frontiers of Computer Science, 18(6): 186345, December 2...

2024 arXiv
[37]

React: Synergizing Reasoning and Acting in Language Models

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing Reasoning and Acting in Language Models. In International Conference on Learning Representations (ICLR), 2023

2023
[38]

Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory, June 2023

Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, Yu Qiao, Zhaoxiang Zhang, and Jifeng Dai. Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based ...

2023 arXiv
[41]

Formally, it calculates the number of instances where: argmax y′∈Y ′ { 1 |a′| Σ a∈a′ Pcf (y′ | a, s′)} ̸= y Where y is the abstraction value observation

Abstraction Change Rate: This metric calculates the proportion of instances where the most probable counterfactual abstraction value differs from the observed abstraction value. Formally, it calculates the number of instances where: argmax y′∈Y ′ { 1 |a′| Σ a∈a′ Pcf (y′ | a, s...
[42]

Counterfactual Probability Increase Rate: This metric measures the proportion of cases where the counterfactual probability for the observed abstraction value is greater than its interventional probability. Formally, it calculates the number of instances where: Pcf (y | a′, s′...
[43]

Sex and Shared Desires,

Semantic Tightness: We also evaluate the semantic similarity between different counter- factual samples generated from the same factual setting. Formally, given a set of strings a = {a1, a2, . . . , an} and a semantic embedding model λ, we can compute the semantic tightness as...
[2014]

arXiv:1412.2985 [cs]

URL http://arxiv.org/abs/1412.2985. arXiv:1412.2985 [cs]

arXiv
[2024]

arXiv:2402.09631 [cs]

URL http://arxiv.org/abs/2402.09631. arXiv:2402.09631 [cs]

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Challenges and Opportunities in Text Generation Explainability

Kenza Amara, Rita Sevastjanova, and Mennatallah El-Assady. Challenges and Opportunities in Text Generation Explainability, May 2024. URL http://arxiv.org/abs/2405.08468. arXiv:2405.08468 [cs]

work page Pith review arXiv 2024

[2] [2]

Constructing Effective Personalized Policies Using Counterfactual Inference from Biased Data Sets with Many Features

Onur Atan, William R. Zame, Qiaojun Feng, and Mihaela van der Schaar. Constructing Effective Personalized Policies Using Counterfactual Inference from Biased Data Sets with Many Features, July 2018. URL http://arxiv.org/abs/1612.08082. arXiv:1612.08082 [stat]

work page Pith review arXiv 2018

[3] [3]

Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)

Leo Breiman. Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author). Statistical Science, 16(3):199–231, 08 2001. doi: 10.1214/ss/1009213726. URL http://dx.doi.org/10.1214/ss/1009213726

arXiv 2001

[4] [4]

RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control, July 2023

Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnik...

arXiv 2023

[5] [5]

Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search

Lars Buesing, Theophane Weber, Yori Zwols, Sebastien Racaniere, Arthur Guez, Jean-Baptiste Lespiau, and Nicolas Heess. Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search. arXiv preprint arXiv:1811.06272, 2018

arXiv 2018

[6] [6]

Ruth M. J. Byrne. Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pages 6276–6282, Macao, China, August 2019. International Joint Conferences on Artificial Intelligence Organization. ISBN 978-0-9992411-4-1. doi: 10.24...

work page 2019

[7] [7]

Counterfactual Token Generation in Large Language Models, September 2024

Ivi Chatzi, Nina Corvelo Benz, Eleni Straitouri, Stratis Tsirtsis, and Manuel Gomez-Rodriguez. Counterfactual Token Generation in Large Language Models, September 2024. URL http: //arxiv.org/abs/2409.17027. arXiv:2409.17027 [cs]

arXiv 2024

[8] [8]

Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting, January 2019

Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexan- dra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting, January 2019. URL http://arxiv.org/abs/1901.09451. arXiv:1901.09451

arXiv 2019