Causality in the human niche: lessons for machine learning

Konrad P. Kording; Richard D. Lange

REVIEW 3 major objections 5 minor 8 references

Causality in the human niche: lessons for machine learning

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper contends that the causality of the human niche—the everyday world of a social, autonomous, goal-driven agent—differs in kind from what Structural Causal Models capture, and argues that machine learning should adopt human-like…

desk verdict A well-argued position paper that gives a useful ecological framing for causal ML; the main soft spot is that the transfer-from-humans-to-machines premise is plausible but not earned, which is a gap for a roadmap, not a fatal flaw. read the letter →

arxiv 2506.13803 v1 pith:RPK6OEUS submitted 2025-06-13 cs.AI cs.LG

Richard D. Lange , Konrad P. Kording This is my paper · ORCID

classification cs.AIcs.LG

keywords causalitymachinelearninghumannichestructuralcausalmodelsinductioncoarse-grainingcuriosityinductivebias

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the causal reasoning humans deploy every day is adapted to what it calls the 'human niche'—the environment, constraints, and goals of a social, autonomous, goal-driven agent—and that this kind of causality is qualitatively different from the kind captured by Structural Causal Models (SCMs). The authors contend that SCM assumptions such as tabular variable sets, acyclic directed graphs, independent exogenous noise, and modular interventions are well suited to carefully designed experiments and expert-curated datasets, but break down in the open-ended, confounded, agent-relative settings where humans actually live and act. If the argument is right, machine learning should build causal systems around human-like inductive biases—causal theories that instantiate new models on the fly, analogy over object ontology, coarse-graining in space and time, sparsity, and curiosity—to make AI more capable, controllable, and interpretable.

What carries the argument

The central mechanism is the 'human niche' concept: an ecological analysis that maps properties of the environment, constraints, and goals onto the causal competencies those properties select for (the paper's Table 1). The argument works by pairing each SCM assumption (acyclicity, no unobserved confounders, modularity, statistical independence of mechanisms, tabular variables) with a niche property that renders that assumption inappropriate, and by proposing the human-like inductive biases—causal theories, analogy over ontology, coarse-graining, sparsity, curiosity, amortized inference—as design principles for ML agents in analogous niches.

What would settle it

A benchmark study in which a learner with explicit causal-theory priors, object ontology, and coarse-graining is pitted against a standard SCM learner on sparse, confounded, open-ended tasks under a fixed data budget; if the human-like learner does not show measurably better zero-shot transfer, the paper's central recommendation is directly contradicted. A behavioral counterpart would test whether people's causal judgments in niche-like tasks track interventional success more closely than observational correlation, as the adaptiveness premise predicts.

Watch

Extended reading notes

Core claim

The central claim is that causality is operational, contextual, and subjective: a causal model is a finite agent's tool for deciding what to do next, not an omniscient description of the true world. In the human niche, the world presents agency, complexity, open-endedness, confounding, human design, other agents, hierarchical and ontological structure, sparse interactions, a continuous data stream, and an arrow of time. Human causal cognition adapts to those properties through good-enough models, causal induction from causal theories, coarse-graining, sparsity and dyadic atomicity, epistemic humility, curiosity and hypothesis-driven exploration, and mental simulation. The paper then reviews the core assumptions of Structural Causal Models—tabular variables, directed acyclic graphs, independent exogenous noise, modular and statistically independent mechanisms, and identifiability as the goal—and argues that each is either unnecessary or counterproductive for an agent in the human niche.

Load-bearing premise

The argument depends on the premise that human causal cognition is genuinely adaptive to the human niche, and that the same inductive biases will transfer to artificial agents; if either gives way, the call to build human-like biases into machine learning loses its foundation.

Editorial extensions

If this is right

Causal evaluation should shift from recovering a true graph to measuring interventional usefulness for an embodied agent.
Causal representation learning should treat object ontology as a fundamental structure, since it enables analogical zero-shot transfer.
Coarse-graining across space and time should be a first-class inductive bias, not an afterthought, for hierarchical and multi-scale modeling.
Curiosity and hypothesis-driven exploration belong inside causal learning: targeted interventions can resolve confounding more cheaply than passive observation.
The independence-of-mechanisms assumption should be relaxed in favor of sharing causal structure across ontologically similar objects.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test of the paper's premise would compare a human-like causal learner (object ontology, causal theories, coarse-graining) with a standard SCM learner on a benchmark of sparse, confounded, open-ended tasks; the paper points to this but does not run it.
The argument implicitly supports treating large language models as repositories of human-niche causal theories: the framing suggests that their zero-shot generalization may be a machine implementation of causal induction, and that this can be probed by intervention-based tasks.
If the adaptiveness premise is correct, then apparent human 'causal illusions'—reading cause into simultaneity or moving objects—are not bugs but niche-tuned heuristics, which would refocus robustness research from truthfulness of models to utility of models.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read it. It's a position paper, not a result paper, and as position papers go it's a good one. The new thing is the systematic mapping of human causal competencies to properties of the 'human niche' — environment, constraints, goals — and then using that map to critique the SCM framework and propose ML research directions. Table 1 is genuinely useful; the SCM critique is fair and actually more careful than most, especially the distinction between interventionally and statistically independent causal mechanisms. The authors know the literature and don't oversell what's known.

The soft spots are the ones you'd expect. The central claim that human-like biases will help ML in the human niche rests on an equivocation between 'humans are adapted to their niche' and 'these biases are good design choices for any agent in that niche.' The paper doesn't earn the second claim. It's plausible, but the evidence cited only shows the first. Human working-memory limits may make coarse-graining and dyadic sparsity necessary, not optimal; some of the biases they cite (like the simultaneity 'causal illusion') are errors, not obviously useful heuristics. The Discussion does exempt well-controlled settings but gives no criterion for when a setting is 'analogous enough.' This is a genuine gap, but it's the gap of a roadmap, not a fatal flaw — the paper is upfront that it's offering hypotheses.

What the paper does well is organize a lot of material into a coherent, falsifiable-in-principle research agenda. It's not a takedown of SCMs, it's a scoping argument. The math is nonexistent, but that's fine for what it is. Citation pattern is broad and honest; self-citations are not load-bearing.

If you're working on causal representation learning, interpretability, or agent design, this is worth a read and worth citing as a framing document. I'd send it to peer review — it deserves a serious referee, and the referee's main job should be to push the authors to sharpen the transfer argument and maybe add a concrete benchmark or example.

Referee Report

3 major / 5 minor

Summary. This position paper argues that the problem of causality for agents operating in the 'human niche' differs fundamentally from the kind of causality captured by Structural Causal Models (SCMs), and that machine learning should therefore import human-like inductive biases—causal theories, analogy, coarse-graining, sparsity, curiosity—to build more capable, controllable, and interpretable systems. The paper proceeds in two parts: Section 2 reviews properties of the human niche (environment, constraints, goals) and maps them to features of human causal cognition, summarized in Table 1; Section 3 critically reviews core SCM assumptions (DAG structure, modularity, independent mechanisms, identifiability) and argues that they are poorly suited to open-ended, agent-relative, ontology-rich settings. The Discussion acknowledges that SCMs remain useful in well-controlled settings. The paper is an argumentative review with no new experiments or formalism.

Significance. If the central claim is accepted, it challenges a substantial research program in causal ML by suggesting that the field's dominant abstractions (static DAGs, variables defined a priori, modular mechanisms, identifiability as the goal) are mismatched to the settings where generalist agents actually operate. The paper is valuable as a broad, readable synthesis: it assembles a wide range of cognitive science and ML work, organizes it under a clear ecological framework (Table 1 and Table 2 are useful reference tools), and offers a nuanced critique that acknowledges where SCMs are successful (tabular medical, economic, and social-science problems). The authors are appropriately cautious in the Discussion, noting that agents in factories may not need curiosity or causal induction. The paper also deserves credit for pointing to concrete research directions (causal representation learning, amortized inference over theories, object-centric models) that are already active.

major comments (3)

[Section 2, opening paragraph] The load-bearing premise of the paper is the sentence 'It follows that models deployed in contexts analogous to the human niche will benefit from analogous inductive biases and cognitive characteristics.' This is an equivocation between two senses of adaptiveness. The evidence marshalled in Section 2 supports the claim that human causal biases are adaptive for humans—e.g., dyadic sparsity and coarse-graining are tied to limited attention and working memory (Section 2.2.3), and causal theories help under sparse data (Section 2.1.8). But the recommended ML transfer requires the stronger claim that these biases are near-optimal, or at least beneficial, for any bounded agent in that niche. Working-memory limits make sparsity a constraint, not a demonstrated optimal solution; the paper itself cites 'causal illusions' (Section 2.1.11) that are systematic errors, and the billiard-ball example (Section 2.1.9) shows a reference-frame-dependent bias that is not obviously performance-enhancing. The Discussion's exemption of well-controlled settings (Section 4) does not supply a criterion for when a deployment context is 'analogous enough' to the human niche. Without such a criterion or a concrete test, the central recommendation remains an analogical leap rather than a consequence of the niche analysis.
[Section 3.2 and 3.3] The critical review of SCMs overstates its case by neglecting existing temporal and cyclic extensions. The billiard-ball and domino examples (Figure 2) are presented as showing that SCMs 'fail to capture' intuitive causal dynamics, but the paper itself acknowledges that an unrolled SCM can represent the physics ('such a model is cumbersome to use,' Section 2.1.9). The paper does not engage with the substantial literature on dynamic Bayesian networks, time-indexed SCMs, or cyclic SCMs (the latter is cited in Bongers et al. 2021 but not discussed). The conclusion that a single static DAG is awkward for these examples is fair, but the broader claim that the SCM framework lacks the resources to capture these aspects of human causal cognition is not established. This matters because the critique of SCMs is the paper's primary motive for looking beyond SCMs; an overstated critique weakens the argument's foundation.
[Section 3.5 and abstract] The paper identifies the variable-construction problem as central ('for any given problem, where do the variables themselves come from?'), but the proposed human-inspired remedies (causal theories, ontology, analogy) are not given any formal or algorithmic content. The paper cites existing work such as typed causal discovery (Brouillard et al. 2022) and theory-based RL (Tsividis et al. 2021) but does not critically evaluate their performance or explain how they would scale to open-ended settings. The abstract claims that leveraging human-like inductive biases will create 'more capable, controllable, and interpretable systems,' yet no experimental evidence, worked example, or falsifiable prediction is offered. For a position paper this may be acceptable, but then the manuscript should be framed as a research proposal with open questions, rather than as a conclusion supported by the niche analysis.

minor comments (5)

[Throughout] There are numerous typos and misspellings: 'structual' (Section 1), 'by through' (Section 2.1.3), 'Y et' (Section 2.2.5), 'Countefactuals' (Section 3.4), 'casual self-consistency' (Section 2.1.7, presumably 'causal'), 'a lightswitch' (Section 2.1.5), and 'Infromation' in the Stuhlmueller et al. bibliography entry. A careful proofread is needed.
[Bibliography] Several references are incomplete or inconsistent: Bareinboim 2020 ('Causal Reinforcement Learning') has no venue, Cohen 2022 has no year or venue, and some entries mix preprint and published formats. The paper also cites 'Xia et al. 2021' in the text but the reference is to a NeurIPS paper with a different author order in the bibliography; please verify all citations.
[Section 2.1.11] The example of a loud sound and a light clicking off (Siegel 2011) is described as an 'illusion,' but the text could clarify whether the illusion is in the simultaneity cue or in the directionality of causality; as written it is slightly ambiguous.
[Section 3.4] The point that CBNs with unobserved variables can support counterfactuals is a useful corrective, but the sentence 'There is no strict requirement that conditional probabilities are constructed by deterministic mechanisms' could be expanded to make the argument more accessible to readers trained on standard SCM counterfactual recipes.
[Table 2] In the row 'Independent exogenous noise,' the footnote marks counterfactuals as enabled by this assumption, but the text of Section 3.4 correctly notes that independence is not strictly necessary. The table's footnote should be updated to reflect this nuance.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a conceptual synthesis with no fitted predictions and no derivation that reduces to its inputs.

full rationale

This is a position/review paper that presents no fitted parameters, no quantitative predictions, and no equations whose outputs are identified with their inputs. The load-bearing step is conceptual: it infers that human causal biases are adaptive in the human niche and recommends transferring analogous biases to ML. That inference is not circular, because the supporting evidence cited is largely external to the authors and the conclusion is not used to define the premise. The only self-citations (Liu, Ungar and Kording 2021; Silva et al. 2023) are incidental background for quasi-experimental design and lifelong learning, and neither is load-bearing in the sense of being the unique justification for a central claim; the argument would stand without them. The skeptic's concern about a possible equivocation between 'adaptive for humans' and 'near-optimal for arbitrary agents in the same niche' is a correctness or validity concern, not an instance of the paper reducing a prediction to its own input by construction. No circularity is therefore present.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper is a conceptual review; its central claims rest on domain assumptions from cognitive science and on a transfer argument from human cognition to ML, rather than on free parameters or new entities. These assumptions are plausible but not empirically established within the paper.

assumptions (3)

domain assumption Human causal cognition is adaptive, meaning it is well-tuned to the structure of the human niche.
Section 2 opens with 'Human cognition is adapted to the environment in which humans evolved...'; the entire argument relies on this premise, which is not independently established in the paper.
domain assumption Inductive biases that help humans will also help ML agents in human-like niches.
Stated in Section 2: 'models deployed in contexts analogous to the human niche will benefit from analogous inductive biases.' This transfer assumption is central but not empirically validated.
domain assumption The SCM framework's standard assumptions (no unobserved confounders, modularity, S-ICM) fail in human-niche settings.
Section 3 argues this through examples like billiard balls and dominoes; the claim is arguable and some SCM extensions exist, so it is an unproven assumption in the argument.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Causality in the human niche: lessons for machine learning." pith.science (2026). https://pith.science/paper/RPK6OEUS

@misc{pith2026250613803,
  author       = {Pith},
  title        = {Pith review of: Causality in the human niche: lessons for machine learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RPK6OEUS}},
  note         = {Machine review of arXiv:2506.13803}
}

read the original abstract

Humans interpret the world around them in terms of cause and effect and communicate their understanding of the world to each other in causal terms. These causal aspects of human cognition are thought to underlie humans' ability to generalize and learn efficiently in new domains, an area where current machine learning systems are weak. Building human-like causal competency into machine learning systems may facilitate the construction of effective and interpretable AI. Indeed, the machine learning community has been importing ideas on causality formalized by the Structural Causal Model (SCM) framework, which provides a rigorous formal language for many aspects of causality and has led to significant advances. However, the SCM framework fails to capture some salient aspects of human causal cognition and has likewise not yet led to advances in machine learning in certain critical areas where humans excel. We contend that the problem of causality in the ``human niche'' -- for a social, autonomous, and goal-driven agent sensing and acting in the world in which humans live -- is quite different from the kind of causality captured by SCMs. For example, everyday objects come in similar types that have similar causal properties, and so humans readily generalize knowledge of one type of object (cups) to another related type (bowls) by drawing causal analogies between objects with similar properties, but such analogies are at best awkward to express in SCMs. We explore how such causal capabilities are adaptive in, and motivated by, the human niche. By better appreciating properties of human causal cognition and, crucially, how those properties are adaptive in the niche in which humans live, we hope that future work at the intersection of machine learning and causality will leverage more human-like inductive biases to create more capable, controllable, and interpretable systems.

Figures

Figures reproduced from arXiv: 2506.13803 by the authors.

**Figure 1.** A) A Structural Causal Model (SCM) expresses causal dependencies between variables {x1,x2,x3,x4} in terms of deterministic functions {f1,f2,f3,f4} and unknown or variable exogenous noise sources {ϵ1,ϵ2,ϵ3,ϵ4}. B) Causal Bayes Nets (CBNs) are a closely-related type of probabilistic graphical model that expresses causal relations between parents and children in terms of conditional probabilities. the noise terms {ϵ1,.… view at source ↗

**Figure 2.** Two simple examples where causal intuitions strain the SCM formalism. A–C) Example 1: An elastic billiard ball collision (example inspired by Cheng and Lu (2017)). A) From the reference frame of ball B, ball A “launches” ball B (or, ball B “stops” ball A). B) The same event viewed from a reference frame initially moving with ball A. Here, ball B “launches” ball A (or, ball A “stops” ball B). C) An accurate but cumbe… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

8 extracted references · 8 canonical work pages

[1]

2.1.7 above

Humans compress complex scenes with numerous complex entities into smaller sets of coarse-grained entities and temporally-extended events; see Sec. 2.1.7 above. Coarse-graining can thus be seen as an adaptation simultaneously to the hierarchical complexity of the environment itself as well as the limitations of finite agents reasoning about a complex world

work page
[2]

2.1.9 above

Humans focus on small numbers of objects or entities at a time, often attending to dyadic relations between just two (possibly coarse-grained) objects; see Sec. 2.1.9 above

work page
[3]

For example, consider the various kinds of causal stories an object like abicyclecan participate in

Humans focus on the relevant properties of those objects or entities as demanded by the present situation. For example, consider the various kinds of causal stories an object like abicyclecan participate in. When planning a commute, one can focus on a bicycle’s properties as a vehicle, ignoring its mechanistic details. When repairing a broken chain, one c...

work page 1995
[4]

Importantly, predictive power does not necessarily require a detailed mechanistic understanding

Useful models include those that make precise and accurate predictions on future data. Importantly, predictive power does not necessarily require a detailed mechanistic understanding. A classic example is the ideal gas law, orPV=nRT, which has high predictive power for how a gas will behave in various settings — both observational and intervening onP,V, o...

work page
[5]

Take Newton’s theory of gravity and Freud’s psychoanalytic theory as examples

Useful models include those which provide insight into underlying mechanisms. Take Newton’s theory of gravity and Freud’s psychoanalytic theory as examples. Both are scientific theories which seek to explain some phenomenon (gravitation, human behavior) in terms of some underlying mechanistic process (gravitational force that obeys an inverse square law, ...

work page
[6]

theory of mind

Useful models include those that balance moderate errors with simplicity and ease-of-use. A model must, by definition, provide some abstraction. The simpler and easier-to-use a model is, the more its inaccuracies can be tolerated. Humans often get by with simple heuristics rather than detailed internal models in situations where the added computational bu...

work page 2024
[2018]

Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search

“Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search.”. Byrne, Ruth M. J. 2016. “Counterfactual Thought.”Annual Review of Psychology67(Volume 67, 2016):135–157. Carey, Susan. 1987.Conceptual Change in Childhood. A Bradford Book 1. mit pr. paperback ed ed. Cambridge, Mass.: MIT Pr. Carey, Susan. 2000. “The Origin of Concepts.”Journal of Cognitio...

work page 2016
[2020]

Bayesian Structure Learning with Generative Flow Networks

Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration. InAdvances in Neural Information Processing Systems. Vol. 33 Curran Associates, Inc. pp. 3761–3774. Cully, Antoine, Jeff Clune, Danesh Tarapore and Jean-Baptiste Mouret. 2015. “Robots That Can Adapt like Animals.”Nature 521(7553):503–507. Danks, David. 2009. The Psychology of C...

work page Pith review arXiv 2015

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

2.1.7 above

Humans compress complex scenes with numerous complex entities into smaller sets of coarse-grained entities and temporally-extended events; see Sec. 2.1.7 above. Coarse-graining can thus be seen as an adaptation simultaneously to the hierarchical complexity of the environment itself as well as the limitations of finite agents reasoning about a complex world

work page

[2] [2]

2.1.9 above

Humans focus on small numbers of objects or entities at a time, often attending to dyadic relations between just two (possibly coarse-grained) objects; see Sec. 2.1.9 above

work page

[3] [3]

For example, consider the various kinds of causal stories an object like abicyclecan participate in

Humans focus on the relevant properties of those objects or entities as demanded by the present situation. For example, consider the various kinds of causal stories an object like abicyclecan participate in. When planning a commute, one can focus on a bicycle’s properties as a vehicle, ignoring its mechanistic details. When repairing a broken chain, one c...

work page 1995

[4] [4]

Importantly, predictive power does not necessarily require a detailed mechanistic understanding

Useful models include those that make precise and accurate predictions on future data. Importantly, predictive power does not necessarily require a detailed mechanistic understanding. A classic example is the ideal gas law, orPV=nRT, which has high predictive power for how a gas will behave in various settings — both observational and intervening onP,V, o...

work page

[5] [5]

Take Newton’s theory of gravity and Freud’s psychoanalytic theory as examples

Useful models include those which provide insight into underlying mechanisms. Take Newton’s theory of gravity and Freud’s psychoanalytic theory as examples. Both are scientific theories which seek to explain some phenomenon (gravitation, human behavior) in terms of some underlying mechanistic process (gravitational force that obeys an inverse square law, ...

work page

[6] [6]

theory of mind

Useful models include those that balance moderate errors with simplicity and ease-of-use. A model must, by definition, provide some abstraction. The simpler and easier-to-use a model is, the more its inaccuracies can be tolerated. Humans often get by with simple heuristics rather than detailed internal models in situations where the added computational bu...

work page 2024

[7] [2018]

Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search

“Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search.”. Byrne, Ruth M. J. 2016. “Counterfactual Thought.”Annual Review of Psychology67(Volume 67, 2016):135–157. Carey, Susan. 1987.Conceptual Change in Childhood. A Bradford Book 1. mit pr. paperback ed ed. Cambridge, Mass.: MIT Pr. Carey, Susan. 2000. “The Origin of Concepts.”Journal of Cognitio...

work page 2016

[8] [2020]

Bayesian Structure Learning with Generative Flow Networks

Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration. InAdvances in Neural Information Processing Systems. Vol. 33 Curran Associates, Inc. pp. 3761–3774. Cully, Antoine, Jeff Clune, Danesh Tarapore and Jean-Baptiste Mouret. 2015. “Robots That Can Adapt like Animals.”Nature 521(7553):503–507. Danks, David. 2009. The Psychology of C...

work page Pith review arXiv 2015