REVIEW 2 major objections 6 minor 20 references
Robot Pouring: Identifying Causes of Spillage and Selecting Alternative Action Parameters Using Probabilistic Actual Causation
T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A probability-raising condition for actual causation can identify which robot pouring parameter caused a spill and select a changed value that prevents the spill in most reruns.
desk verdict First real application of probabilistic actual causation to robot action selection, with a solid pipeline but an evaluation that doesn't yet prove the method beats simpler baselines. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is a probabilistic actual-causation inequality, in which $X = x$ rather than $X = x'$ is the actual cause of $Y = y$ when $P(Y \mid do(W = w^*, X = x, Z' = z^*)) > P(Y \mid do(W = w^*, X = x'))$ holds for every subset $Z'$ of the mediators along the chosen path. $W$ is the set of off-path variables, held at their observed values, so the test is context-specific rather than averaged. Feeding continuous contrastive values through this inequality yields a curve of spill probabilities for next actions; the automatic search then identifies the range where probability raising holds, restricts to values with predicted spill probability below a threshold, and picks the value closest to the current parameter. The DAG structure and the conditional distributions needed for the do-probabilities are learned from simulation data, with Monte Carlo integration over the estimated conditional distributions answering each query.
What would settle it
Run a controlled experiment in the same simulator where a hidden variable, for example marble diameter, varies between two distributions while all measured parameters are held at identical values; if the spill rate shifts systematically with the hidden variable, the DAG is not causally sufficient and the alternative parameters chosen by the actual-causation test should fail to transfer. A second check would be to compare the method's selected alternative value against a brute-force grid search over all parameter combinations in the simulator on the same spill trials.
Extended reading notes
Core claim
The paper's central claim is that the probabilistic actual-causation inequality can serve both as a diagnostic and as a prescription in a concrete manipulation task. In the learned causal graph, spillage has direct parents relative volume, fullness, and relative diameter, with relative capacity acting indirectly through relative volume; for a given spill trial the analysis fixes off-path variables and mediators at their observed values, then compares the reference probability $P(S \mid do(W = w^*, X = x, Z' = z^*))$ with $P(S \mid do(W = w^*, X = x'))$ over a grid of contrastive values $x'$. When probability raising holds, $x$ is an actual cause of that spill relative to $x'$, and the range of $x'$ where it holds contains candidate corrective values. The paper selects the value closest to the current one with predicted spill probability below 0.1, and shows that rerunning the simulated trials with those selected values succeeds most of the time, while the same causal probabilities averaged over contexts give much weaker and less actionable guidance. In short: the inequality identifies what to change and how much to change, in the context of the particular observed trial.
Load-bearing premise
The load-bearing premise is that the learned causal graph of the pouring task is the true causal structure, with no unobserved common causes; if a hidden factor jointly affects a task parameter and spillage, the interventional probabilities and probability-raising tests are biased and the suggested alternative parameter could be wrong.
Editorial extensions
If this is right
- When the graph and probability estimates are reliable, the actual-causation search can automate corrective action selection after each failed trial, without random retries.
- Direct causes are the most actionable: an alternative relative-diameter value was found in 97.9% of spill trials, an alternative fullness value in 59%, and an alternative relative-capacity value in only 2.7%.
- The chosen alternatives produced empirical success rates of 88.7% for relative diameter and 86.9% for fullness, all above chance and in line with the probability threshold used.
- Because the variable effects are non-linear, a small change in a parameter can separate a spilling from a successful pour, and the actual-causation scan finds such changes where human intuition might miss them.
- Average causal probabilities are not enough: the causal probability of spillage given relative capacity stays near chance level, while the context-conditional actual-cause comparison reveals when and how much to change a parameter.
Reading between the lines
- If this transfers to physical robots with accurate perception of container dimensions and fullness, the same inequality could choose corrective actions after failed manipulation attempts without random retries; that transfer is a testable extension the paper has not run.
- The binary spillage variable discards severity, so a natural extension is a graded outcome such as number or fraction of particles spilled; the actual-causation test could then rank alternatives by expected severity reduction, and the authors themselves note that severity matters in some application contexts.
- Because the graph is assumed causally sufficient, deployment would benefit from a robustness layer that compares the selected alternative against any other parameter change that also reduces predicted spill probability, so a misspecified graph cannot silently steer the correction.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a complete pipeline for using Fenton-Glynn's probabilistic actual causation to diagnose spillage in a simulated robot pouring task and to select alternative values of task parameters. A causal DAG (RC, FU, RV, RD, S) is learned from simulated trials with the PC algorithm, conditional distributions are estimated with neural autoregressive density estimators, and the actual-causation inequality (1) is applied to test direct (RD) and mediated (FU) causes. On a held-out test set of 3000 trials (1216 spillage trials), an alternative RD value was identified in 1191 trials with 88.7% success when rerun, and an alternative FU value in 718 trials with 86.9% success. Four worked examples illustrate the context-sensitive selection behavior. The authors claim this goes beyond type-level causal probabilities because the analysis conditions on the actual values of other variables.
Significance. If correct, the paper provides one of the first practical implementations of token-level probabilistic actual causation for robotic action correction, and it demonstrates a complete causal-modeling pipeline (causal discovery, NADE-based do-probability estimation, and held-out simulation evaluation). Strengths include the use of a realistic physics simulation covering a large parameter space, bootstrapped structure learning, correct application of the Fenton-Glynn inequalities for both direct and mediated paths, and a held-out evaluation with a confusion matrix. The main unresolved issue is that the evaluation does not compare the proposed criterion against simpler baselines, so the paper's central claim that actual causation adds value beyond type-level causal probabilities is not yet established.
major comments (2)
- [Section 5.3; Figures 5e and 10] The evaluation reports only absolute success rates for the selected alternatives. Because the simulation is stochastic, a spillage trial may succeed on a simple retry; the retry success rate for the 1216 spillage trials is not reported. More importantly, the paper does not compare the proposed token-level selection with (a) changing the parameter to any low-probability value chosen by a type-level criterion such as P(S|do(RD)) below a threshold (which Section 5.1, Figure 5e, shows has a large low-probability region for RD greater than about 1.1), (b) a context-free threshold on the NADE prediction P(S|FU,RD,RV), or (c) changing a variable not identified as an actual cause. Without these baselines, the high success rates in Section 5.3 do not demonstrate that the actual-causation probability-raising test, rather than the low-probability threshold or the NADE predictions, is responsible for the performance. The authors should add these comparisons and report how often the probability-raising condition changes the selected value relative to a threshold-only rule.
- [Sections 4.4-4.5 and 5.1; inequalities (10)-(11)] The NADE models P(RV|RC,FU) as a Gaussian distribution, but in the data-generating process RV is a nearly deterministic function of FU and RC (Table 2), and the paper itself notes in Section 5.1 that the NADE for RV is smoothing the estimated probabilities. Because the FU actual-causation inequalities (10) and (11) integrate over P(RV|RC,FU), this smoothing can bias the probability-raising comparisons and therefore the conclusion that FU is an actual cause. The paper should quantify the approximation error (e.g., compare the NADE-implied conditional distribution with empirical conditional frequencies or use a conditional model reflecting the known near-determinism) and report sensitivity of the FU results to this choice.
minor comments (6)
- [Section 5.3] The phrase 'alternative DD values' is a typo and should read 'alternative RD values'.
- [Section 5.3] The sentence 'Running these trials with the alternative F Uvalues success rate of 86.9%' is missing a verb and should be 'produced a success rate of 86.9%'.
- [Section 5.3] Report 95% confidence intervals for the success rates (88.7% and 86.9%) and for the alternative-identification rates (97.9% for RD and 59% for FU); the histograms in Figure 10 help but only cover 100 trials.
- [Section 5.3] State explicitly that for FU the probability-raising test requires both inequalities (10) and (11), and report how often each is satisfied; the current text says only that 'probability raising holds' without distinguishing the two conditions.
- [Sections 4.1 and 5.3] Clarify the relationship between the 6000 simulated trials mentioned in Section 4.1 and the 3000-trial test set in Section 5.3, i.e., whether the training and test sets are disjoint and how many trials were used for training.
- [Section 5.1] The term 'chance level' is used without a definition; specify that it refers to a spillage probability of 0.5.
Circularity Check
No significant circularity: the actual-causation analysis is applied as stated, and the alternative-action success rates are evaluated on a separate physics-simulation test set rather than re-stating the model's own inputs.
full rationale
The derivation chain runs from (i) a DAG discovered with the PC algorithm on simulated trials (Fig. 3), (ii) NADE-estimated conditional distributions (Eq. 3), (iii) Fenton-Glynn probability-raising inequalities (Eqs. 9-11), to (iv) alternative-parameter selection using a fixed 0.1 probability threshold (Sections 2.1 and 5.3), and (v) empirical evaluation on a separate 3000-trial simulation test set. No load-bearing input is defined in terms of the target claim: the graph is learned from data rather than imposed to match the spillage outcome; the conditional distributions are estimated on training data and then used to select alternative RD/FU values; the actual-causation inequalities are applied exactly as stated by Fenton-Glynn and do not contain a fitted parameter later renamed as a prediction; and the reported 88.7% and 86.9% success rates are obtained by re-running the physics simulation with the suggested alternative values, which is an independent benchmark rather than a re-statement of the model's own probability estimates. The acknowledged limitations (no latent variables assumed, no ground-truth causal graph, binary spillage representation, and the absence of baselines such as random restarts or type-level causal-probability criteria) are correctness or external-validity concerns, not circularity. There are no load-bearing self-citations: the cited methodological works are external, and no uniqueness theorem from the present authors is invoked to force the modeling choice. The paper is therefore self-contained against an external simulation benchmark, and the correct circularity finding is 0.
Assumptions & free parameters
free parameters (3)
- probability threshold for alternative selection =
0.1 (evaluation), 0.2 and 0.5 (examples)
- causal discovery edge-frequency threshold =
0.5 for stable edges, 0.75 to construct the reported DAG
- NADE hyperparameters =
2 hidden layers of 16 units, RMSProp, learning rate 0.01
assumptions (5)
- domain assumption The PC-discovered DAG with edge frequencies above the bootstrap threshold is the true causal structure (RC->RV, FU->RV, RV->S, FU->S, RD->S), with no unobserved confounders.
- domain assumption The CoppeliaSim/ODE simulation is a faithful proxy for real robot pouring dynamics.
- ad hoc to paper Choosing a contrastive value where probability raising holds and predicted spillage probability is below threshold is a valid corrective action.
- domain assumption The NADE conditional densities accurately represent the true conditional distributions over the support of the data.
- domain assumption Continuous variables RC, FU, RV, and RD are well approximated by Gaussian conditional distributions.
Cite this review
Pith. "Pith review of Robot Pouring: Identifying Causes of Spillage and Selecting Alternative Action Parameters Using Probabilistic Actual Causation." pith.science (2026). https://pith.science/paper/QTYNJP4Y
@misc{pith2026250209395,
author = {Pith},
title = {Pith review of: Robot Pouring: Identifying Causes of Spillage and Selecting Alternative Action Parameters Using Probabilistic Actual Causation},
year = {2026},
howpublished = {\url{https://pith.science/paper/QTYNJP4Y}},
note = {Machine review of arXiv:2502.09395}
}
read the original abstract
In everyday life, we perform tasks (e.g., cooking or cleaning) that involve a large variety of objects and goals. When confronted with an unexpected or unwanted outcome, we take corrective actions and try again until achieving the desired result. The reasoning performed to identify a cause of the observed outcome and to select an appropriate corrective action is a crucial aspect of human reasoning for successful task execution. Central to this reasoning is the assumption that a factor is responsible for producing the observed outcome. In this paper, we investigate the use of probabilistic actual causation to determine whether a factor is the cause of an observed undesired outcome. Furthermore, we show how the actual causation probabilities can be used to find alternative actions to change the outcome. We apply the probabilistic actual causation analysis to a robot pouring task. When spillage occurs, the analysis indicates whether a task parameter is the cause and how it should be changed to avoid spillage. The analysis requires a causal graph of the task and the corresponding conditional probability distributions. To fulfill these requirements, we perform a complete causal modeling procedure (i.e., task analysis, definition of variables, determination of the causal graph structure, and estimation of conditional probability distributions) using data from a realistic simulation of the robot pouring task, covering a large combinatorial space of task parameters. Based on the results, we discuss the implications of the variables' representation and how the alternative actions suggested by the actual causation analysis would compare to the alternative solutions proposed by a human observer. The practical use of the analysis of probabilistic actual causation to select alternative action parameters is demonstrated.
Figures
Figures from the paper (12 more)
Reference graph
Works this paper leans on
-
[1]
The Actual Cause, chapter 10, pages 309–330
Judea Pearl. The Actual Cause, chapter 10, pages 309–330. Cambridge University Press, Cambridge, September 2009a. ISBN 9780521749190. doi: 10.1017/cbo9780511803161.012. Sander Beckers. Causal explanations and XAI. In Bernhard Schölkopf, Caroline Uhler, and Kun Zhang, editors, Proceedings of the First Conference on Causal Learning and Reasoning, volume 177...
-
[2]
the poured amount (fullness and volume, expressed as the variables F Uand RV ) on the probability of spilling (variable S). The randomness of the outcome S results from the interplay between RC, RD, F U, and RV and the behavior of the particles during the pouring movement. The parameters the physics engine uses (particle size and density) to simulate the ...
work page 2021
-
[7]
Ramaravind Kommiya Mothilal, Divyat Mahajan, Chenhao Tan, and Amit Sharma
doi: 10.48550/ARXIV .2401.13752. Ramaravind Kommiya Mothilal, Divyat Mahajan, Chenhao Tan, and Amit Sharma. Towards unifying feature attribution and counterfactual explanations: Different means to the same end. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’21, pages 652–663. ACM, July
-
[10]
ISSN 2377-3774. doi: 10.1109/lra.2022.3188889. Eric Rohmer, Surya P. N. Singh, and Marc Freese. V-rep: A versatile and scalable robot simulation framework. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, November
arXiv 2022
-
[13]
Note that within-tier edges are allowed in the background knowledge. To validate the causal relationships inferred from the data, we performed a bootstrapping analysis. Conducting a bootstrapping analysis is recommended for causal discovery[Malinsky and Danks, 2017, Glymour et al., 2019]. We ran the PC algorithm on 1000 bootstraps, producing 1000 differen...
work page 2017
-
[14]
ISSN 1942-4795. doi: 10.1002/widm.1449. Daniel Malinsky and David Danks. Causal discovery algorithms: A practical guide. Philosophy Compass, 13(1), November
-
[15]
ISSN 1747-9991. doi: 10.1111/phc3.12470. Joseph D Ramsey, Kun Zhang, Madelyn Glymour, Ruben Sanchez Romero, Biwei Huang, Imme Ebert-Uphoff, Savini Samarasinghe, Elizabeth A Barnes, and Clark Glymour. Tetrad—a toolbox for causal discovery. In 8th international workshop on climate informatics,
-
[16]
ISSN 2193-3685. doi: 10.1515/jci-2020-0007. Santtu Tikka and Juha Karvanen. Identifying causal effects with the R package causaleffect. Journal of Statistical Software, 76(12):1–30,
Show all 20 references
-
[17]
doi: 10.18637/jss.v076.i12. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamku...
-
[18]
URL https://proceedings.neurips.cc/paper_files/paper/2019/file/ bdbca288fee7f92f2bfa9f7012727740-Paper.pdf. Supplementary Material 7.1 Causal discovery using the PC algorithm - Tetrad setup and bootstrapping results Figure 12 shows the exact parameters used in Tetrad to run th...
2019
-
[1991]
doi: 10.1177/089443939100900106
ISSN 1552-8286. doi: 10.1177/089443939100900106. Clark Glymour, Kun Zhang, and Peter Spirtes. Review of causal discovery methods based on graphical models. Frontiers in Genetics, 10, June
-
[2005]
doi: 10.1093/bjps/axi147
ISSN 1464-3537. doi: 10.1093/bjps/axi147. Jan Borner. Causal explanations - how to generate, identify, and evaluate them. PhD thesis,
-
[2013]
www.coppeliarobotics.com
doi: 10.1109/iros.2013.6696520. www.coppeliarobotics.com. Peter Spirtes and Clark Glymour. An algorithm for fast recovery of sparse causal graphs. Social Science Computer Review, 9(1):62–72, April
2013
-
[2017]
doi: 10.1093/ bjps/axv056
ISSN 1464-3537. doi: 10.1093/ bjps/axv056. Judea Pearl. Causal Diagrams and the Identification of Causal Effects, chapter 3, pages 65–106. Cambridge University Press, September 2009b. ISBN 9780521749190. doi: 10.1017/cbo9780511803161.005. Maximilian Diehl and Karinne Ramirez-A...
-
[2019]
doi: 10.3389/fgene.2019.00524
ISSN 1664-8021. doi: 10.3389/fgene.2019.00524. 21 Robot Pouring - Probabilistic Actual Causation Ana Rita Nogueira, Andrea Pugnana, Salvatore Ruggieri, Dino Pedreschi, and João Gama. Methods and tools for causal discovery and causal inference. WIREs Data Mining and Knowledge D...
2019
-
[2020]
A causal approach to tool affordance learning
Jake Brawer, Meiying Qin, and Brian Scassellati. A causal approach to tool affordance learning. In 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 8394–8399. IEEE,
2020
-
[2021]
Ossama Ahmed, Frederik Träuble, Anirudh Goyal, Alexander Neitz, Yoshua Bengio, Bernhard Schölkopf, Manuel Wüthrich, and Stefan Bauer
doi: 10.1145/3461702.3462597. Ossama Ahmed, Frederik Träuble, Anirudh Goyal, Alexander Neitz, Yoshua Bengio, Bernhard Schölkopf, Manuel Wüthrich, and Stefan Bauer. Causalworld: A robotic manipulation benchmark for causal structure and transfer learning. arXiv preprint arXiv:20...
-
[2022]
doi: 10.1007/978-3-031-24670-8_9
ISBN 9783031246708. doi: 10.1007/978-3-031-24670-8_9. Ehsan Zibaei and Robin Borth. Building causal models for finding actual causes of unmanned aerial vehicle failures. Frontiers in Robotics and AI, 11, February
-
[2023]
doi: 10.1016/j.robot
ISSN 0921-8890. doi: 10.1016/j.robot. 2023.104376. Hugo Araujo, Patrick Holthaus, Marina Sarda Gou, Gabriella Lakatos, Giulia Galizia, Luke Wood, Ben Robins, Mohammad Reza Mousavi, and Farshid Amirabdollahian. Kaspar Causally Explains, pages 85–99. Springer Nature Switzerland,
2023
-
[2024]
doi: 10.3389/frobt.2024.1123762
ISSN 2296-9144. doi: 10.3389/frobt.2024.1123762. Hana Chockler, Daniel Kroening, and Youcheng Sun. Explanations for occluded images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 1234–1243, October
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.