Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

SlotPi: Physics-informed Object-centric Reasoning Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SlotPi embeds Hamiltonian physics into slot-based object-centric models and claims state-of-the-art prediction across rigid bodies, fluids, and their coupled real-world dynamics.

desk verdict SlotPi is a credible incremental extension of STATM with a useful new fluid-object dataset, but the paper does not demonstrate that the Hamiltonian module is what drives the reported gains. read the letter →

arxiv 2506.10778 v1 pith:O3FMIVGS submitted 2025-06-12 cs.CV cs.AIcs.LG

Jian Li , Wan Han , Ning Lin , Yu-Liang Zhan , Ruizhi Chengze , Haining Wang , Yi Zhang , Hongsheng Liu
show 3 more authors
Zidong Wang Fan Yu Hao Sun
This is my paper
classification cs.CVcs.AIcs.LG
keywords object-centriclearningHamiltonianneuralnetworksphysics-informedmachinevideopredictionvisualquestionansweringfluiddynamicsslotattentionworldmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SlotPi is an object-centric video prediction model that writes Hamilton's equations into the latent dynamics of slot representations. It claims that attention over slots can supply learned generalized coordinates and momenta, and that a learned Hamiltonian computed by self-attention can advance the scene through gradient updates. A parallel spatiotemporal attention module is added to capture non-conservative behavior that Hamiltonian dynamics alone cannot represent. If the claim holds, a single slot-based framework improves prediction quality and predictive visual question answering on rigid-object benchmarks, competes with neural operator methods on Navier-Stokes fluid data, and extends to real-world fluid-object interactions; the authors report state-of-the-art results on every dataset tested.

What carries the argument

The load-bearing object is a latent Hamiltonian layer built entirely from slot attention. Cross-attention maps each slot's history to generalized momentum $P_t$, self-attention maps the current slots to generalized coordinates $Q_t$, and a self-attention energy head with Softplus yields $H_t$. The layer then applies Hamilton's equations and an Euler integrator to produce $\hat Q_{t+1}$ and $\hat P_{t+1}$, and the final prediction is $\hat S_{t+1}=\lambda \hat Q_{t+1}+\hat{ST}_{t+1}$, where $\hat{ST}_{t+1}$ is the output of a spatiotemporal attention module. The physics layer is what carries the inductive bias; the residual module is what lets the hybrid handle dissipation and non-conservative forces.

What would settle it

Run a long rollout on a conservative scene with no external forcing, such as OBJ3D, and track the predicted energy $H_t$ from Eq. (2); if energy drifts as much as it does when the physics module is ablated ($\lambda=0$), the Hamiltonian term is adding capacity rather than enforcing physics. A sharper check is to test whether the learned update is approximately symplectic by comparing the Jacobian of $(\hat Q,\hat P)$ with respect to $(Q,P)$ with a symplectic matrix on held-out trajectories.

Watch

Extended reading notes

Core claim

SlotPi's central claim is that constraining slot evolution with a learned Hamiltonian layer makes object-centric dynamics prediction both more accurate and more transferable. Starting from slots $S_t$ extracted by an upstream object-centric model, cross-attention over slot histories yields generalized momenta $P_t$ and self-attention yields generalized coordinates $Q_t$; a self-attention head with Softplus produces per-slot energies whose sum is the Hamiltonian $H_t$. Hamilton's equations $\dot Q_t=\partial H_t/\partial P_t$ and $\dot P_t=-\partial H_t/\partial Q_t$ are integrated with an Euler step to predict $\hat Q_{t+1}$ and $\hat P_{t+1}$, and the final slot prediction is the weighted sum of this physical state and an unconstrained spatiotemporal residual. The authors report that SlotPi outperforms SlotFormer, STATM, FNO, and U-Net on OBJ3D, CLEVRER, Physion-Collide VQA, the PDE-Refiner Navier-Stokes benchmark, and their own real-world fluid-object dataset, with the largest margins on the most interaction-heavy scenes.

Load-bearing premise

The load-bearing premise is that the numbers the attention mechanism computes really are positions and momenta of a physical system, so the model's energy estimate is a true energy whose gradients drive correct motion; the paper does not test conservation or symplectic structure.

Editorial extensions

If this is right

  • Slot-based world models can carry an explicit energy-based inductive bias without knowing the governing equations in advance.
  • The same architecture transfers from rigid-body scenes to pure Navier-Stokes fluid prediction, so the physical constraint need not be task-specific.
  • More physically consistent rollouts translate into higher accuracy on CLEVRER predictive questions and on Physion-Collide, closing part of the gap toward human performance.
  • The real-world fluid-object dataset demonstrates that the model can capture coupled behaviors such as floating objects being pushed, rotated, and held back by fluid motion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the biggest gains are reported on CLEVRER, the most interaction-heavy benchmark, so the Hamiltonian prior may matter most when collisions and rearrangements dominate rather than on simple or slow scenes.
  • Because Eq. (6) adds an unconstrained residual at $\lambda=1$, SlotPi is best understood as a Hamiltonian-regularized hybrid rather than a strictly conservative simulator, and on non-conservative scenes the residual may carry much of the signal.
  • A natural extension would be to restrict the residual module to learned dissipation (for example, a Rayleigh-like damping term), which would make the Hamiltonian constraint testable and recover exact symplectic structure in the conservative limit.
  • The fluid experiments treat image patches as slots rather than discovered objects, suggesting that the object-centric machinery is acting partly as a spatial tokenizer; that reading could connect SlotPi to other PDE surrogate and operator-learning problems.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces SlotPi, a slot-based object-centric dynamics model that combines a Hamiltonian-style physics module (Eqs. 1-4) with a spatiotemporal attention module (Eq. 5) through a weighted sum (Eq. 6). The model is evaluated on video prediction and VQA on OBJ3D, CLEVRER, Physion, on a Navier-Stokes fluid dataset, and on a new real-world fluid-object dataset collected by the authors. The central claims are that SlotPi achieves state-of-the-art results on all datasets and that the Hamiltonian module provides physical constraints that improve predictive accuracy and cross-domain adaptability.

Significance. The paper's potential contribution is a single object-centric framework that unifies rigid-body, fluid, and coupled fluid-object prediction, together with a new real-world dataset and an ablation of Hamiltonian embedding choices. The manuscript is transparent about its two-stage training pipeline, and the limitations section is candid. However, the current evidence does not establish the load-bearing role of the Hamiltonian constraint, and the 'state-of-the-art on all datasets' claim is contradicted by the OBJ3D PSNR result in Table 1. With additional isolation experiments and direct physical-consistency metrics, the contribution could be solid.

major comments (4)
  1. [Section 4.4 / Table 7 / Eq. (6)] The ablation for the physical module is incomplete and does not support the text. The paragraph states that 'When the physical module is removed (i.e., λ=0), the model degrades to a structure similar to STATM or Slotformer, resulting in inferior performance,' but Table 7 reports only λ=1, λ=0.1, and a learnable λ; no λ=0 row is shown. Furthermore, there is no same-capacity non-Hamiltonian update baseline. Because Eq. (6) is a plain sum with λ=1, the unconstrained residual cST could in principle account for the full prediction, so the reported gains over STATM may be due to added capacity rather than to the physics-informed inductive bias. This is the central load-bearing point of the paper and needs a direct ablation.
  2. [Abstract, Contributions, Table 1] The claim that 'SlotPi achieves state-of-the-art results on all datasets' is not supported by Table 1: SAVi-Dyn attains PSNR 32.94 while SlotPi attains 32.67 on OBJ3D, with SSIM tied. Additionally, no standard deviations or numbers of seeds are reported for any experiment. On CLEVRER the PSNR gap over STATM is only 0.19 dB (30.41 vs 30.22), which is within typical run-to-run noise for this class of models; without repeated runs the 'state-of-the-art' performance claim is not statistically grounded.
  3. [Section 3.1-3.2, Eqs. (1)-(6), Section 4.1] The paper does not verify that the attention-computed Q_t and P_t form a valid Hamiltonian phase space. Q_t and P_t are both functions of the same slot representation S_t (Eq. 1), with no symplectic structure or canonical-coordinate test. Moreover, the predicted momentum Phat_{t+1} from Eq. (4b) is discarded and never used in Eq. (6), and the final prediction adds an unconstrained residual cST with λ=1. No energy conservation, symplecticity, or other direct physical-consistency metric is reported. Therefore the statement in Section 4.1 that the model's predictions are 'more physically consistent' is an inference from image-quality metrics, not a demonstrated property.
  4. [Section 4.2, Table 5] The fluid experiments use the PDE-Refiner dataset (ref [43]) but compare only against FNO and U-Net. PDE-Refiner itself, a strong baseline specifically designed for long-rollout accuracy on this benchmark, is omitted. Without this comparison and without error bars, the claim of 'significantly outperforms other models' on the Navier-Stokes task is not fully established.
minor comments (5)
  1. [Section 3, notation] The definition S_t = {s(0,t),...,s(N,t)} ∈ R^{N×D} is inconsistent because it lists N+1 slots while claiming N slots; Eq. (7) sums i=0 to N, which reinforces the inconsistency. Please clarify whether there are N or N+1 slots.
  2. [Section 4.2, metric name] 'High Correction Time (HCT)' appears to be a typo; if it is 'High Correlation Time', please correct and define it (e.g., time until correlation drops below a threshold).
  3. [Section 4.1, Table 3 discussion] 'second rand' should be 'second rank' (or 'second place').
  4. [Section 4.4, Figure 6] Please report the numerical values of the initial and final learned λ and the training epoch count; the description 'gradually increases, but it does not exceed 1' is vague.
  5. [Section 2, Related Work] The related-work section covers many references but does not discuss the relationship to generalized Hamiltonian learning methods (e.g., neural symplectic form) in detail; a brief positioning sentence would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found: SlotPi's predictions are evaluated on held-out data, and the Hamiltonian module is an architectural choice rather than a fitted or self-referential derivation.

full rationale

SlotPi's derivation chain is a conventional supervised autoregressive slot predictor. Slots S_t are obtained from upstream object-centric encoders; the physical module's Q_t, P_t, and H_t are learnable attention functions of these slots; the spatiotemporal residual cST_{t+1} is also a learnable attention function; and the final prediction bS_{t+1} = lambda * bQ_{t+1} + cST_{t+1} is trained by an l2 loss against future slots. No parameter is fit to a subset of the reported test labels and then renamed as a prediction. Equations (3)-(4) impose Hamilton's equations by definition for the learned H_t; this is a structural choice, not a circular derivation, because no external physical quantity (e.g., true energy, symplectic invariant, or conservation law) is claimed to be predicted or validated. The paper does cite its own STATM and STATM-SAVi for slot extraction and spatiotemporal-attention machinery, and the lambda=0 ablation called for in Section 4.4 is not tabulated; these bear on whether the physics module is responsible for the reported gains, but they do not make any reported result equivalent to its input by construction. Evaluations on CLEVRER, OBJ3D, Physion, the NS fluid dataset, and the new real-world dataset are all against held-out data with external baselines. Therefore no circular step is exhibited; the remaining concerns are correctness and attribution issues, not circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central contribution rests on a learned Hamiltonian energy that is fit to the same data, and several hand-set hyperparameters (lambda, Delta t, slot counts) are not reported in full. No new physical entities are introduced, but the latent coordinates and energy are invented structures without independent evidence.

free parameters (4)
  • Delta t (Euler integration step) = not reported; set small based on fps or simulation step
    Equation (4) uses Delta t to integrate Hamilton's equations; the value is chosen by hand per dataset and is not listed.
  • lambda (physics-residual blend weight) = 1.0 for all main experiments; learnable in ablation
    Equation (6) weights the Hamiltonian coordinate prediction against the spatiotemporal residual; the paper fixes lambda=1 outside ablations.
  • Per-dataset slot count and dimension = 7x128 CLEVRER, 6x192 real-world, 1024x512 fluid
    Architecture capacity is set differently per dataset; these choices affect the reported performance but are not systematic.
  • Hamiltonian energy network parameters = learned from data
    The energy H_t in Eq. (2) is a trainable attention-plus-linear network, so the 'physical' prediction is fitted to the training data rather than derived.
assumptions (4)
  • domain assumption Hamilton's equations Qdot = dH/dP and Pdot = -dH/dQ hold in the learned latent space
    Section 3.1 Eq. (3) assumes the attention-computed coordinates and momenta form a Hamiltonian phase space; no experiment verifies this.
  • domain assumption Slots extracted by upstream object-centric models faithfully decompose scenes into distinct entities
    Section 3 states SlotPi operates on slots from SAVi or STATM-SAVi; Figure 5 shows qualitative decomposition but no quantitative guarantee for fluid-object scenes.
  • ad hoc to paper The spatiotemporal residual captures all non-conservative effects outside the Hamiltonian module
    Equations (5) and (6) add an unconstrained learned residual to the physics prediction, with no dissipation model or constraint, so any non-conservative behavior is delegated to the network.
  • standard math Standard automatic differentiation and Euler integration are valid for training and rollout
    The gradients in Eq. (3) and updates in Eq. (4) are implemented with autodiff; no numerical stability analysis is provided.
invented entities (1)
  • Latent generalized coordinates Q_t and momenta P_t
    purpose: To impose Hamiltonian structure on slot dynamics
    These are attention outputs (Eq. 1) with no verified correspondence to physical positions or momenta; no probe or intervention test is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SlotPi: Physics-informed Object-centric Reasoning Models." pith.science (2026). https://pith.science/paper/O3FMIVGS

@misc{pith2026250610778,
  author       = {Pith},
  title        = {Pith review of: SlotPi: Physics-informed Object-centric Reasoning Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O3FMIVGS}},
  note         = {Machine review of arXiv:2506.10778}
}
read the original abstract

Understanding and reasoning about dynamics governed by physical laws through visual observation, akin to human capabilities in the real world, poses significant challenges. Currently, object-centric dynamic simulation methods, which emulate human behavior, have achieved notable progress but overlook two critical aspects: 1) the integration of physical knowledge into models. Humans gain physical insights by observing the world and apply this knowledge to accurately reason about various dynamic scenarios; 2) the validation of model adaptability across diverse scenarios. Real-world dynamics, especially those involving fluids and objects, demand models that not only capture object interactions but also simulate fluid flow characteristics. To address these gaps, we introduce SlotPi, a slot-based physics-informed object-centric reasoning model. SlotPi integrates a physical module based on Hamiltonian principles with a spatio-temporal prediction module for dynamic forecasting. Our experiments highlight the model's strengths in tasks such as prediction and Visual Question Answering (VQA) on benchmark and fluid datasets. Furthermore, we have created a real-world dataset encompassing object interactions, fluid dynamics, and fluid-object interactions, on which we validated our model's capabilities. The model's robust performance across all datasets underscores its strong adaptability, laying a foundation for developing more advanced world models.

Figures

Figures reproduced from arXiv: 2506.10778 by the authors.

Figure 1
Figure 1. (a) Overall framework. SlotPi employs slots extracted by the object-centric model for reasoning and is mainly composed [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Long-sequence prediction results on OBJ3D and CLEVRER. On OBJ3D, STATM performs well, but Slotformer exhibits [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Rollout qualitative results on the NS fluid dataset. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Prediction results on real-world dataset. The fluid flows slowly downward, influencing each object to move slowly [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: The value of the learnable 𝜆 changes throughout the training process. 𝜆0 represents the initial value of 𝜆; N denotes the number of attention blocks in the spatiotemporal reasoning module [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Prediction results of different number of frames [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Smoothing Slot Attention Iterations and Recurrences

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    SmoothSA preheats cold-start Slot Attention queries via self-distillation and differentiates aggregation iterations between first and later video frames, improving several object-centric learning benchmarks.

Reference graph

Works this paper leans on

94 extracted references · 46 canonical work pages · cited by 1 Pith paper

  1. [43]

    Phillip Lippe, Bas Veeling, Paris Perdikaris, Richard Turner, and Johannes Brand- stetter. 2023. Pde-refiner: Achieving accurate long rollouts with neural pde solvers. Advances in Neural Information Processing Systems36 (2023), 67398–67433

    2023

  2. [1]

    Renee Baillargeon. 1987. Object permanence in 31/2-and 41/2-month-old infants. Developmental psychology23, 5 (1987), 655

    1987

  3. [2]

    Yohai Bar-Sinai, Stephan Hoyer, Jason Hickey, and Michael P Brenner. 2019. Learning data-driven discretizations for partial differential equations.Proceedings of the National Academy of Sciences116, 31 (2019), 15344–15349

    2019

  4. [3]

    Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, et al

  5. [4]

    Daniel Bear, Chaofei Fan, Damian Mrowca, Yunzhu Li, Seth Alter, Aran Nayebi, Jeremy Schwartz, Li F Fei-Fei, Jiajun Wu, Josh Tenenbaum, et al. 2020. Learning physical graph representations from visual scenes.Advances in Neural Information Processing Systems33 (2020), 6027–6039

    2020

  6. [5]

    Daniel M Bear, Elias Wang, Damian Mrowca, Felix J Binder, Hsiao-Yu Fish Tung, RT Pramod, Cameron Holdaway, Sirui Tao, Kevin Smith, Fan-Yun Sun, et al. 2021. Physion: Evaluating physical prediction from vision in humans and machines. arXiv preprint arXiv:2106.08261(2021)

    arXiv 2021

  7. [6]

    G Brockman. 2016. OpenAI Gym.arXiv preprint arXiv:1606.01540(2016)

    arXiv 2016

  8. [7]

    Christopher P Burgess, Loic Matthey, Nicholas Watters, Rishabh Kabra, Irina Higgins, Matt Botvinick, and Alexander Lerchner. 2019. Monet: Unsupervised scene decomposition and representation.arXiv preprint arXiv:1901.11390(2019)

    arXiv 2019

Show all 94 references
  1. [8]

    Michael B Chang, Tomer Ullman, Antonio Torralba, and Joshua B Tenenbaum

  2. [9]

    Chang Chen, Fei Deng, and Sungjin Ahn. 2021. Roots: Object-centric representa- tion and rendering of 3d scenes.The Journal of Machine Learning Research22, 1 (2021), 11770–11805

    2021

  3. [10]

    arXiv preprint arXiv:1612.00341(2016)

    A compositional object-based approach to learning physical dynamics. arXiv preprint arXiv:1612.00341(2016)

    2016 arXiv

  4. [11]

    Zhijie Chen, Mingquan Feng, Junchi Yan, and Hongyuan Zha. 2022. Learn- ing neural Hamiltonian dynamics: a methodological overview.arXiv preprint arXiv:2203.00128(2022)

    2022 arXiv

  5. [12]

    Yuhan Chen, Takashi Matsubara, and Takaharu Yaguchi. 2021. Neural symplectic form: Learning Hamiltonian equations on general coordinate systems.Advances in Neural Information Processing Systems34 (2021), 16659–16670

    2021

  6. [13]

    Kevin Course, Trefor Evans, and Prasanth Nair. 2020. Weak form generalized hamiltonian learning.Advances in Neural Information Processing Systems33 (2020), 18716–18726

    2020

  7. [14]

    Andy Clark. 2013. Whatever next? Predictive brains, situated agents, and the future of cognitive science.Behavioral and brain sciences36, 3 (2013), 181–204

    2013

  8. [15]

    Mingyu Ding, Zhenfang Chen, Tao Du, Ping Luo, Josh Tenenbaum, and Chuang Gan. 2021. Dynamic visual reasoning by learning differentiable physics models from video and language.Advances In Neural Information Processing Systems34 (2021), 887–899

    2021

  9. [16]

    David Ding, Felix Hill, Adam Santoro, Malcolm Reynolds, and Matt Botvinick

  10. [17]

    Danny Driess, Zhiao Huang, Yunzhu Li, Russ Tedrake, and Marc Toussaint. 2023. Learning multi-object dynamics with compositional neural radiance fields. In Conference on Robot Learning. PMLR, 1755–1768

    2023

  11. [18]

    Gamaleldin Elsayed, Aravindh Mahendran, Sjoerd van Steenkiste, Klaus Greff, Michael C Mozer, and Thomas Kipf. 2022. Savi++: Towards end-to-end object- centric learning from real-world videos.Advances in Neural Information Processing Systems35 (2022), 28940–28954

    2022

  12. [19]

    Andrea Dittadi, Samuele Papa, Michele De Vita, Bernhard Schölkopf, Ole Winther, and Francesco Locatello. 2021. Generalization and robustness implications in object-centric learning.arXiv preprint arXiv:2107.00637(2021)

    2021 arXiv

  13. [20]

    Alison Gopnik and Henry M Wellman. 1992. Why the child’s theory of mind really is a theory. (1992)

    1992

  14. [21]

    Samuel Greydanus, Misko Dzamba, and Jason Yosinski. 2019. Hamiltonian neural networks.Advances in neural information processing systems32 (2019)

    2019

  15. [22]

    Han Gao, Luning Sun, and Jian-Xun Wang. 2021. PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain.J. Comput. Phys.428 (2021), 110079

    2021

  16. [23]

    Jayesh K Gupta and Johannes Brandstetter. 2022. Towards multi-spatiotemporal- scale generalized pde modeling.arXiv preprint arXiv:2209.15616(2022)

    2022 arXiv

  17. [24]

    Mohammed Hassanin, Saeed Anwar, Ibrahim Radwan, Fahad S Khan, and Ajmal Mian. 2022. Visual attention methods in deep learning: An in-depth survey.arXiv preprint arXiv:2204.07756(2022)

    2022 arXiv

  18. [25]

    Gaurav Gupta, Xiongye Xiao, and Paul Bogdan. 2021. Multiwavelet-based opera- tor learning for differential equations.Advances in neural information processing systems34 (2021), 24048–24062

    2021

  19. [26]

    Quercus Hernández, Alberto Badías, Francisco Chinesta, and Elías Cueto. 2023. Thermodynamics-informed neural networks for physically realistic mixed reality. Computer Methods in Applied Mechanics and Engineering407 (2023), 115912. KDD ’25, August 3–7, 2025, Toronto, ON, Canada...

    2023

  20. [27]

    James Howse, Chaouki Abdallah, and Gregory Heileman. 1995. Gradient and Hamiltonian dynamics applied to learning in neural networks.Advances in Neural Information Processing Systems8 (1995)

    1995

  21. [28]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition. 770–778

    2016

  22. [29]

    Steeven Janny, Aurélien Beneteau, Madiha Nadri, Julie Digne, Nicolas Thome, and Christian Wolf. 2023. Eagle: Large-scale learning of turbulent fluid dynamics with mesh transformers.arXiv preprint arXiv:2302.10803(2023)

    2023 arXiv

  23. [30]

    Jindong Jiang, Sepehr Janghorbani, Gerard De Melo, and Sungjin Ahn. 2019. Scalor: Generative world models with scalable object representations.arXiv preprint arXiv:1910.02384(2019)

    2019 arXiv

  24. [31]

    Lawrence Hubert and Phipps Arabie. 1985. Comparing partitions.Journal of classification2 (1985), 193–218

    1985

  25. [32]

    Rishabh Kabra, Daniel Zoran, Goker Erdogan, Loic Matthey, Antonia Creswell, Matt Botvinick, Alexander Lerchner, and Chris Burgess. 2021. Simone: View- invariant, temporally-abstracted object representations via unsupervised video decomposition.Advances in Neural Information Pr...

    2021

  26. [33]

    Thomas Kipf, Gamaleldin Fathy Elsayed, Aravindh Mahendran, Austin Stone, Sara Sabour, Georg Heigold, Rico Jonschkowski, Alexey Dosovitskiy, and Klaus Greff. 2021. Conditional Object-Centric Learning from Video. InInternational Conference on Learning Representations

    2021

  27. [34]

    Marko Jusup, Petter Holme, Kiyoshi Kanazawa, Misako Takayasu, Ivan Romić, Zhen Wang, Sunčana Geček, Tomislav Lipić, Boris Podobnik, Lin Wang, et al

  28. [35]

    James R Kubricht, Keith J Holyoak, and Hongjing Lu. 2017. Intuitive physics: Current research and controversies.Trends in cognitive sciences21, 10 (2017), 749–759

    2017

  29. [36]

    Adarsh Kumar, Peter Ku, Anuj Goyal, Angeliki Metallinou, and Dilek Hakkani-Tur

  30. [37]

    Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman

  31. [38]

    Adam Kosiorek, Hyunjik Kim, Yee Whye Teh, and Ingmar Posner. 2018. Sequential attend, infer, repeat: Generative modelling of moving objects.Advances in Neural Information Processing Systems31 (2018)

    2018

  32. [39]

    Jian Li, Pu Ren, Yang Liu, and Hao Sun. 2024. Reasoning-Enhanced Object-Centric Learning for Videos. arXiv:2403.15245 [cs.CV] https://arxiv.org/abs/2403.15245

    2024 arXiv

  33. [40]

    Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. 2020. Fourier neural oper- ator for parametric partial differential equations.arXiv preprint arXiv:2010.08895 (2020)

    2020 arXiv

  34. [41]

    Zijie Li, Dule Shu, and Amir Barati Farimani. 2024. Scalable transformer for pde surrogate modeling.Advances in Neural Information Processing Systems36 (2024)

    2024

  35. [42]

    Zhixuan Lin, Yi-Fu Wu, Skand Peri, Bofeng Fu, Jindong Jiang, and Sungjin Ahn

  36. [44]

    Kookjin Lee, Nathaniel Trask, and Panos Stinis. 2021. Machine learning structure preserving brackets for forecasting irreversible processes.Advances in Neural Information Processing Systems34 (2021), 5696–5707

    2021

  37. [45]

    Zichao Long, Yiping Lu, Xianzhong Ma, and Bin Dong. 2018. Pde-net: Learning pdes from data. InInternational conference on machine learning. PMLR, 3208– 3216

    2018

  38. [46]

    Haoyu Lu, Guoxing Yang, Nanyi Fei, Yuqi Huo, Zhiwu Lu, Ping Luo, and Mingyu Ding. 2023. Vdt: General-purpose video diffusion transformers via mask modeling. arXiv preprint arXiv:2305.13311(2023)

    2023 arXiv

  39. [47]

    Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karni- adakis. 2021. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators.Nature machine intelligence3, 3 (2021), 218–229

    2021

  40. [48]

    Chuizheng Meng, Sungyong Seo, Defu Cao, Sam Griesemer, and Yan Liu. 2022. When physics meets machine learning: A survey of physics-informed machine learning.arXiv preprint arXiv:2203.16797(2022)

    2022 arXiv

  41. [49]

    In International conference on machine learning

    Improving generative imagination in object-centric world models. In International conference on machine learning. PMLR, 6140–6149

  42. [50]

    Luis S Piloto, Ari Weinstein, Peter Battaglia, and Matthew Botvinick. 2022. In- tuitive physics learning in a deep-learning model inspired by developmental psychology.Nature human behaviour6, 9 (2022), 1257–1267

    2022

  43. [51]

    Zichao Long, Yiping Lu, and Bin Dong. 2019. PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network.J. Comput. Phys.399 (2019), 108925

    2019

  44. [52]

    Maziar Raissi, Alireza Yazdani, and George Em Karniadakis. 2020. Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations.Science 367, 6481 (2020), 1026–1030

    2020

  45. [53]

    William M Rand. 1971. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical association66, 336 (1971), 846–850

    1971

  46. [54]

    Chengping Rao, Pu Ren, Yang Liu, and Hao Sun. 2022. Discovering nonlinear PDEs from scarce data with physics-encoded learning.arXiv preprint arXiv:2201.12354 (2022)

    2022 arXiv

  47. [55]

    Chengping Rao, Pu Ren, Qi Wang, Oral Buyukozturk, Hao Sun, and Yang Liu

  48. [56]

    Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W Battaglia

  49. [57]

    Learning mesh-based simulation with graph networks.arXiv preprint arXiv:2010.03409(2020)

    2020 arXiv

  50. [58]

    Ronan Riochet, Mario Ynocente Castro, Mathieu Bernard, Adam Lerer, Rob Fergus, Véronique Izard, and Emmanuel Dupoux. 2018. Intphys: A framework and benchmark for visual intuitive physics reasoning.arXiv preprint arXiv:1803.07616 (2018)

    2018 arXiv

  51. [59]

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. 2019. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Computa- tional physics378 (2019), 686–707

    2019

  52. [60]

    Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. 2020. Learning to simulate complex physics with graph networks. InInternational conference on machine learning. PMLR, 8459–8468

    2020

  53. [61]

    H Sebastian Seung, Tom Richardson, J Lagarias, and John J Hopfield. 1997. Mini- max and Hamiltonian dynamics of excitatory-inhibitory networks.Advances in neural information processing systems10 (1997)

    1997

  54. [62]

    Richard M Shiffrin and Walter Schneider. 1977. Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory.Psychological review84, 2 (1977), 127

    1977

  55. [63]

    Gautam Singh, Yi-Fu Wu, and Sungjin Ahn. 2022. Simple unsupervised object- centric learning for complex and naturalistic videos.Advances in Neural Infor- mation Processing Systems35 (2022), 18181–18196

    2022

  56. [64]

    2019.The promise of artificial intelligence: reckoning and judgment

    Brian Cantwell Smith. 2019.The promise of artificial intelligence: reckoning and judgment. Mit Press

    2019

  57. [65]

    Pu Ren, Chengping Rao, Yang Liu, Zihan Ma, Qi Wang, Jian-Xun Wang, and Hao Sun. 2023. PhySR: Physics-informed deep super-resolution for spatiotemporal data.J. Comput. Phys.492 (2023), 112438

    2023

  58. [66]

    Pu Ren, Chengping Rao, Yang Liu, Jian-Xun Wang, and Hao Sun. 2022. PhyCRNet: Physics-informed convolutional-recurrent network for solving spatiotemporal PDEs.Computer Methods in Applied Mechanics and Engineering389 (2022), 114399

    2022

  59. [67]

    Kimberly Stachenfeld, Drummond B Fielding, Dmitrii Kochkov, Miles Cranmer, Tobias Pfaff, Jonathan Godwin, Can Cui, Shirley Ho, Peter Battaglia, and Al- varo Sanchez-Gonzalez. 2021. Learned coarse models for efficient turbulence simulation.arXiv preprint arXiv:2112.15275(2021)

    2021 arXiv

  60. [68]

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolu- tional networks for biomedical image segmentation. InMedical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings...

    2015

  61. [69]

    Kejun Tang, Jiayu Zhai, Xiaoliang Wan, and Chao Yang. 2023. Adversarial adaptive sampling: Unify PINN and optimal transport for the approximation of PDEs.arXiv preprint arXiv:2305.18702(2023)

    2023 arXiv

  62. [70]

    Rishi Veerapaneni, John D Co-Reyes, Michael Chang, Michael Janner, Chelsea Finn, Jiajun Wu, Joshua Tenenbaum, and Sergey Levine. 2020. Entity abstraction in visual model-based reinforcement learning. InConference on Robot Learning. PMLR, 1439–1456

    2020

  63. [71]

    Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Yu

  64. [72]

    Rui Wang, Robin Walters, and Rose Yu. 2020. Incorporating symmetry into deep dynamics models for improved generalization.arXiv preprint arXiv:2002.03061 (2020)

    2020 arXiv

  65. [73]

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing13, 4 (2004), 600–612

    2004

  66. [74]

    Andrew Sosanya and Sam Greydanus. 2022. Dissipative hamiltonian neural networks: Learning dissipative and conservative dynamics separately.arXiv preprint arXiv:2201.10085(2022)

    2022 arXiv

  67. [75]

    Elizabeth S Spelke and Katherine D Kinzler. 2007. Core knowledge.Developmental science10, 1 (2007), 89–96

    2007

  68. [76]

    Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long

  69. [77]

    Aleksandar Stanić and Jürgen Schmidhuber. 2019. R-sqair: Relational sequential attend, infer, repeat.arXiv preprint arXiv:1910.05231(2019)

    2019 arXiv

  70. [78]

    Charig Yang, Hala Lamdouar, Erika Lu, Andrew Zisserman, and Weidi Xie. 2021. Self-supervised video object segmentation by motion grouping. InProceedings of the IEEE/CVF International Conference on Computer Vision. 7177–7188

    2021

  71. [79]

    Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B Tenenbaum. 2019. Clevrer: Collision events for video representation and reasoning.arXiv preprint arXiv:1910.01442(2019)

    2019 arXiv

  72. [80]

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang

  73. [81]

    InProceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining

    Towards physics-informed deep learning for turbulent flow prediction. InProceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 1457–1466

  74. [84]

    Nicholas Watters, Daniel Zoran, Theophane Weber, Peter Battaglia, Razvan Pascanu, and Andrea Tacchetti. 2017. Visual interaction networks: Learning a physics simulator from video.Advances in neural information processing systems 30 (2017)

    2017

  75. [85]

    Marissa A Weis, Kashyap Chitta, Yash Sharma, Wieland Brendel, Matthias Bethge, Andreas Geiger, and Alexander S Ecker. 2020. Unmasking the inductive biases of unsupervised object representations for video sequences.arXiv preprint arXiv:2006.070342 (2020). SlotPi: Physics-inform...

    2020 arXiv

  76. [88]

    Ziyi Wu, Nikita Dvornik, Klaus Greff, Thomas Kipf, and Animesh Garg. 2023. SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models. InThe Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=TFbwV6I0VLg

    2023

  77. [93]

    Daniel Zoran, Rishabh Kabra, Alexander Lerchner, and Danilo J Rezende. 2021. Parts: Unsupervised segmentation with slots, attention and independence maxi- mization. InProceedings of the IEEE/CVF International Conference on Computer Vision. 10439–10447. KDD ’25, August 3–7, 202...

    2021

  78. [192]

    Other training strategies are almost consistent with those used on other datasets

    We subsample the video by a factor of 2 and train for 60 epochs using the Adam optimizer, with a batch size of 64 and a learning rate of2× 10−4. Other training strategies are almost consistent with those used on other datasets. STATM.STATM [39] comprises two main components: a...

    2025

  79. [2016]

    Advances in neural information processing systems29 (2016)

    Interaction networks for learning about objects, relations and physics. Advances in neural information processing systems29 (2016)

    2016

  80. [2017]

    Building machines that learn and think like people.Behavioral and brain sciences40 (2017), e253

    2017

  81. [2018]

    InProceedings of the IEEE conference on computer vision and pattern recognition

    The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition. 586–595

  82. [2020]

    InProceedings of the AAAI conference on artificial intelligence, Vol

    Ma-dst: Multi-attention-based scalable dialog state tracking. InProceedings of the AAAI conference on artificial intelligence, Vol. 34. 8107–8114

  83. [2021]

    Advances in neural information processing systems34 (2021), 9112–9124

    Attention over learned object embeddings enables complex visual reasoning. Advances in neural information processing systems34 (2021), 9112–9124

    2021

  84. [2022]

    Social physics.Physics Reports948 (2022), 1–148

    2022

  85. [2023]

    Encoding physics to learn reaction–diffusion processes.Nature Machine Intelligence5, 7 (2023), 765–779

    2023

  86. [2024]

    Transolver: A fast transformer solver for pdes on general geometries.arXiv preprint arXiv:2402.02366(2024)

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.