Pith. sign in

REVIEW 5 major objections 6 minor 3 cited by

Dyn-O: Building Structured World Models with Object-Centric Representations

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Dyn-O shows object-centric world models can beat monolithic DreamerV3 on complex Procgen games.

desk verdict Object-centric slots plus Mamba dynamics beat a frozen-tokenizer DreamerV3 on Procgen, but the headline comparison is not clean and the abstract overreaches. read the letter →

arxiv 2507.03298 v1 pith:ZMZHHLHO submitted 2025-07-04 cs.LG

Zizhao Wang , Kaixin Wang , Li Zhao , Peter Stone , Jiang Bian This is my paper
classification cs.LG
keywords worldmodelsobject-centricrepresentationsslotattentionstate-spaceMambaProcgenstatic-dynamicdisentanglementrolloutprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dyn-O is an object-centric world model designed to work in visually rich, procedurally generated game environments, not just the simple-shape scenes where such models have mostly been tested. The paper argues that by first learning slot-based object representations with guidance from segmentation masks, then modeling transitions with a state-space dynamics model over those slots, an agent can predict future frames more accurately than a monolithic world model. On seven Procgen games, Dyn-O reports lower LPIPS and FVD and higher SSIM than DreamerV3, and it does so with fewer latent slots than a patch-level alternative. The paper further claims that splitting each object slot into static and dynamic features lets the model change an object's appearance while leaving its motion intact, which supports controlled generation of diverse imagined trajectories.

What carries the argument

The central machinery is segmentation-mask-guided slot attention over frozen pretrained tokenizer features: each slot attends only to image patches inside a SAM2-provided object mask during training, and the mask is gradually dropped so the encoder works without it at inference. Dynamics are modeled by a permutation-equivariant self-attention step followed by a shared Mamba state-space model per slot, with separate prediction heads for the next slots, reward, and termination. The static-dynamic disentanglement is carried by a time-invariance plus InfoNCE loss for static features and a reconstruction plus Wasserstein adversarial loss that drives dynamic features to carry minimal static information.

What would settle it

Evaluate Dyn-O and SOLV on a set of scenes whose object masks come from an independent source (for example human annotation) rather than from SAM2; if Dyn-O's foreground adjusted Rand index advantage over SOLV largely disappears on those masks, then the object-discovery claim reduces to segmentation-mask distillation, not general object-centric discovery.

Watch

Extended reading notes

Core claim

The central claim is that object-centric world models are viable in complex visual environments and can outperform monolithic ones at predicting rollouts. Dyn-O learns object slots from pixels using a frozen pretrained image tokenizer plus slot attention, with a segmentation mask guiding the attention during training and a logarithmic dropout schedule removing the mask at inference. A Mamba state-space model then predicts the next set of slots, and the reported experiments show that this object-slot dynamics model beats DreamerV3 on rollout quality in seven Procgen games, including a drop in average FVD from 692.5 to 361.3 and LPIPS from 0.42 to 0.33. The paper also claims that decoupling each slot into dynamics-agnostic static features and dynamics-aware dynamic features, trained with time-invariance and mutual-information penalties, supports swapping static attributes between objects to produce dynamically consistent 30-step rollouts.

Load-bearing premise

The object-binding evaluation assumes the SAM2 segmentation masks used during training are correct ground-truth objects, so the reported gains in object discovery may mostly show how faithfully Dyn-O copies those masks rather than how well it finds objects on its own.

Editorial extensions

If this is right

  • Object-centric world models can be trained directly from pixels in cluttered, textured environments, not only in toy domains with basic shapes.
  • Predicting in object-slot space can be both more accurate and more compact than predicting in patch-level latent space, since 31 or 47 object slots outperform 196 patch slots.
  • World model rollouts can be manipulated at the object level, for example by swapping static features to change colors while preserving motion.
  • The mask-dropout schedule keeps inference cheap, since the segmentation model is not needed when the trained encoder is deployed.
  • The design separates representation learning from dynamics learning, making it possible to upgrade either component independently.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The FG-ARI object-binding results compare against SAM2 masks used during training, so the reported advantage over fully unsupervised slot learning may partly measure distillation fidelity rather than object discovery from scratch; a more decisive test would use held-out masks or human annotations not derived from SAM2.
  • Because Dyn-O's encoder relies on a frozen pretrained tokenizer and segmentation model, the 'from pixels' claim is conditioned on these foundation-model priors; a natural extension is testing how much of the gain survives when those priors are trained on different domains or removed.
  • The static-dynamic split suggests a concrete use the paper does not demonstrate: generating counterfactual training data for offline reinforcement learning by swapping appearances across trajectories while preserving dynamics.
  • The same architecture could be probed in partially observable or 3D environments, where the slot structure might help track occluded objects across longer rollouts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Dyn-O, an object-centric world model that combines Cosmos tokenizer features with SAM2-mask-guided slot attention and a Mamba state-space dynamics model, and further disentangles each slot into static and dynamic components. The method is evaluated on seven Procgen environments, where the authors report higher rollout prediction accuracy than a DreamerV3 baseline and a patch-based ablation, improved slot binding over SOLV, and evidence for static-dynamic disentanglement.

Significance. If validated, Dyn-O would be a useful step toward object-centric world models in visually complex environments, since most prior work has been confined to simple shapes. The paper has several strengths: it leverages strong frozen visual encoders/decoders, introduces a practical mask-dropout schedule to avoid SAM2 inference overhead, evaluates on unseen levels, and reports standard video metrics. The controlled Dyn-O w/o OC ablation is a useful comparison, and the probing protocol for disentanglement is thoughtful. However, the central claim that Dyn-O outperforms DreamerV3 is not cleanly supported: the DreamerV3 baseline is handicapped by a frozen Cosmos tokenizer, the caveflyer results partly contradict the blanket claim, and the FG-ARI evaluation appears to be at least partly circular. These issues, while fixable, are load-bearing for the paper's headline conclusions.

major comments (5)
  1. [Abstract, Section 4.2, Table 1] The claim that Dyn-O 'outperforms DreamerV3 in rollout prediction accuracy' is not supported by the experimental design as presented. The authors state that DreamerV3 is run 'with the same frozen Cosmos tokenizer,' which prevents DreamerV3 from learning its own task-adapted encoder, discrete latent space, and decoder from pixels. This likely handicaps the baseline. The controlled Dyn-O w/o OC ablation is the right comparison for the benefit of object-centric structure; the DreamerV3 comparison should be presented as an additional, clearly qualified result rather than as the basis for the abstract's blanket claim.
  2. [Table 6, caveflyer row] The per-environment results contradict the statement in Section 4.2 that 'Dyn-O significantly outperforms dreamer': on caveflyer, Dyn-O is worse than DreamerV3 on LPIPS (0.61 vs 0.53), SSIM (0.26 vs 0.44), and PSNR (10.82 vs 11.99). The aggregate Table 1 therefore masks a real failure mode. Please qualify the claim to reflect per-environment performance and discuss possible reasons (e.g., object count, dynamics, or segmentation quality in caveflyer).
  3. [Section 4.1, Figure 4, Table 5] The FG-ARI evaluation compares slot masks against masks that appear to be the same SAM2 masks used to supervise training. If this is the case, the metric largely measures fidelity to the teacher model, not independent object discovery. The paper should explicitly state the source of the 'ground-truth' masks and, ideally, report FG-ARI against an independent annotation source or a synthetic environment with known object masks. The low Oracle coinrun value (0.33) also indicates that SAM2 masks may be noisy, which further undermines their use as ground truth without clarification.
  4. [Section 3.2 and Section 4.2] Since the paper claims improved dynamics modeling over prior object-centric world models, the absence of any comparison to SlotSSM, Dreamweaver, or other object-centric world models leaves this claim incomplete. Please either add an adapted baseline on Procgen or clearly state why those methods cannot be applied to these settings, with supporting evidence rather than an assertion.
  5. [Section 3.1 and Appendix A.1] The mask-dropout schedule is described only as log(1 + # updates)/log(1 + # total updates), but the exact schedule parameters (start probability, end probability, and the point at which the mask is fully dropped) are not given, and no sensitivity analysis is provided. Since the ability to run without SAM2 at inference is a core contribution, this detail needs to be fully specified and, ideally, ablated.
minor comments (6)
  1. [Section 4.2] The phrase 'accurate and and generalizable' in Section 4.2 contains a typo and should read 'accurate and generalizable.'
  2. [Table 5] The table header uses 'FR-ARI' while the text in Section 4.1 uses 'FG-ARI'; please make the acronym consistent.
  3. [Table 5] The environment name 'dogeball' in Table 5 is misspelled; it should be 'dodgeball' as elsewhere in the paper.
  4. [Reference [38]] The reference title contains a typo, 'Regularing', which should be 'Regularizing'.
  5. [Equation (2)] The loss in Eq. (2) uses a fixed horizon T for both the slot prediction and reward/termination terms; please clarify how variable-length episodes are handled in the summation bounds.
  6. [Table 1] Table 1 reports only aggregate means without standard errors; please include per-environment standard errors or confidence intervals, as in Table 6, so the reader can assess variability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation chain is self-contained and externally evaluated; the one self-citation is not load-bearing.

full rationale

The paper's derivation chain—learning object-centric slots from Cosmos features with SAM2-guided slot attention, then training a Mamba SSM to predict slot transitions—does not reduce any claimed result to its own input. The rollout comparison against DreamerV3 is against an external baseline, and the Dyn-O w/o OC ablation isolates the object-centric grouping while keeping the same Cosmos encoder/decoder and SSM backbone. The FG-ARI evaluation is not shown to be circular: the paper never states that the ground-truth masks are the same SAM2 masks used in training, and the Oracle baseline, which explicitly uses SAM2 masks at inference, scores only 0.33 on coinrun in Table 5, which is inconsistent with evaluating against those very masks. The static-dynamic probing is partly a consistency check of the time-invariance and adversarial disentanglement objectives, but the informative results—static features retaining color and dynamic features predicting position—are not forced by construction. The only self-citation is [24] (CAGrad), used in Appendix A.2 to balance slot reconstruction and disentanglement gradients; it is an implementation detail and is not load-bearing. Concerns about DreamerV3 being handicapped by a frozen Cosmos tokenizer are experimental-design issues, not circularity, and do not change the circularity verdict.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central claim rests on pretrained Cosmos and SAM2 models, per-environment slot count K, and losses that encourage disentanglement. None of these are derived from first principles. The evaluation partially leans on the same SAM2 masks used for training.

free parameters (2)
  • Slot count K per environment = 31 (bigfish, coinrun, dodgeball, jumper), 47 (ninja, starpilot), 63 (caveflyer)
    Hyperparameter chosen per environment, not learned; slot count must roughly match object count, and mismatches can leave empty slots or split objects.
  • Mask dropout schedule shape = log(1 + updates) / log(1 + total_updates)
    The schedule controlling how quickly SAM2 supervision is removed is chosen by hand, and its shape affects the final quality of slot binding.
assumptions (6)
  • domain assumption SAM2-generated segmentation masks provide correct object boundaries in Procgen.
    Section 3.1 uses SAM2 masks as attention masks during training, and Section 4.1 uses them as the ground truth for FG-ARI.
  • domain assumption Frozen Cosmos tokenizer features preserve enough visual information for slot extraction and future-frame decoding.
    Section 3.1 uses CosmosEnc and CosmosDec frozen; if these features discard details, all downstream predictions inherit that loss.
  • domain assumption Slot Attention with reconstruction pressure yields meaningful object slots after mask-guided pretraining.
    Section 3.1 relies on the unsupervised slot attention objective to maintain object binding once the mask is dropped.
  • domain assumption Mamba state-space models can capture object interaction dynamics across slots.
    Section 3.2 applies a shared SSM to each slot and relies on self-attention between slots to model interactions.
  • domain assumption InfoNCE and adversarial discrimination reduce mutual information between static and dynamic features enough for true disentanglement.
    Section 3.2 uses these losses to separate static and dynamic components, but the paper cites Locatello et al. to note that unsupervised disentanglement is generally hard.
  • domain assumption DreamerV3 with a frozen Cosmos tokenizer is a fair proxy for DreamerV3's capability.
    Section 4.2 compares against this variant; if the frozen tokenizer handicaps DreamerV3, the headline result would be weaker.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dyn-O: Building Structured World Models with Object-Centric Representations." pith.science (2026). https://pith.science/paper/ZMZHHLHO

@misc{pith2026250703298,
  author       = {Pith},
  title        = {Pith review of: Dyn-O: Building Structured World Models with Object-Centric Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZMZHHLHO}},
  note         = {Machine review of arXiv:2507.03298}
}
read the original abstract

World models aim to capture the dynamics of the environment, enabling agents to predict and plan for future states. In most scenarios of interest, the dynamics are highly centered on interactions among objects within the environment. This motivates the development of world models that operate on object-centric rather than monolithic representations, with the goal of more effectively capturing environment dynamics and enhancing compositional generalization. However, the development of object-centric world models has largely been explored in environments with limited visual complexity (such as basic geometries). It remains underexplored whether such models can generalize to more complex settings with diverse textures and cluttered scenes. In this paper, we fill this gap by introducing Dyn-O, an enhanced structured world model built upon object-centric representations. Compared to prior work in object-centric representations, Dyn-O improves in both learning representations and modeling dynamics. On the challenging Procgen games, we find that our method can learn object-centric world models directly from pixel observations, outperforming DreamerV3 in rollout prediction accuracy. Furthermore, by decoupling object-centric features into dynamics-agnostic and dynamics-aware components, we enable finer-grained manipulation of these features and generate more diverse imagined trajectories.

Figures

Figures reproduced from arXiv: 2507.03298 by the authors.

Figure 1
Figure 1. A high-level overview of the object-centric (OC) world model framework. The latent features [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Components of Dyn-O: (a) object-centric representation learning; (b) dynamics learning. Modules marked with are fixed, while others are learnable. The "Dyn-O Encoder" in (b) corresponds to the lower half of (a), which maps the image o to the latent slot feature z. See Section 3 for details. frame o ∈ RH×W×3 , Dyn-O first applies the Cosmos encoder (CosmosEnc) to extract patch-level features f ∈ R N×df , where N deno… view at source ↗
Figure 3
Figure 3. Illustration of (a) the overall design for disentangling slot features in dynamics modeling, and the training procedures for (b) static features and (c) dynamic features. // indicates a stop-gradient operation. 3.2 World Model with Object-Centric Representations After learning object-centric representations as described above, the next phase of Dyn-O focuses on training a world model to reason about object interacti… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Evaluation of the object-centric representation learning in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Dyn-O generates more accurate rollouts than Dreamer in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Dyn-O generates dynamically consistent rollouts after exchanging static features. In the top two rows, we swap the static features of the avatar (the small agent in the center of the image) between two initial states and generate 30-step rollouts using Dyn-O. The resul…
Figure 7
Figure 7. Figure 7: Qualitative evaluation of the object-centric representation learning in [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: 20-step rollouts in dodgeball. 1st row: ground-truth, 2nd row: Dyn-O (ours), 3rd row: DreamerV3, and 4th row: Dyn-O w/o OC. Dyn-O significantly outperforms dreamer, with sharp player shape and accurate predictions of threw balls [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: 20-step rollouts in jumper. 1st row: ground-truth, 2nd row: Dyn-O (ours), 3rd row: DreamerV3, and 4th row: Dyn-O w/o OC. Dyn-O significantly outperforms dreamer, with sharp wall and player trail until 10th timestamp [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: 20-step rollouts in ninja. 1st row: ground-truth, 2nd row: Dyn-O (ours), 3rd row: DreamerV3, and 4th row: Dyn-O w/o OC. Dyn-O significantly outperforms dreamer, with sharper wall shape at 15-th timestamp. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry

    cs.LG 2026-04 accept novelty 7.0 of 10

    WAV self-improves action-conditioned world models by cycle-consistent verification of state plausibility and sparse action reachability, doubling sample efficiency and lifting policy reward by over 22% on nine tasks.

  2. Factored Latent Action World Models

    cs.LG 2026-02 conditional novelty 6.0 of 10

    FLAM splits a scene into separate factors, each with its own latent action, and reports better video prediction and downstream policy learning than monolithic latent-action models.

  3. A Comprehensive Survey on World Models for Embodied AI

    cs.CV 2025-10 conditional novelty 6.0 of 10

    A unified three-axis taxonomy — functionality, temporal modeling, spatial representation — organizes the world-model literature for embodied AI.

Reference graph

Works this paper leans on

45 extracted references · 24 canonical work pages · cited by 3 Pith papers

  1. [1]

    Self-supervised Object-centric Learning for Videos

    Aydemir, G., Xie, W., and Güney, F. Self-supervised Object-centric Learning for Videos. InAdvances in Neural Information Processing Systems, 2023

    work page 2023

  2. [2]

    Dreamweaver: Learning compositional world representa- tions from pixels

    Baek, J., Wu, Y.-F., Singh, G., and Ahn, S. Dreamweaver: Learning compositional world representa- tions from pixels. arXiv preprint arXiv: 2501.14174, 2025

    arXiv 2025

  3. [3]

    Bai, J., Wang, W., and Gomes, C. P . Contrastively disentangled sequential variational autoencoder. Advances in Neural Information Processing Systems, 34:10105–10118, 2021

    work page 2021

  4. [4]

    Sequential disentanglement by extracting static information from a single sequence element

    Berman, N., Naiman, I., Arbiv, I., Fadlon, G., and Azencot, O. Sequential disentanglement by extracting static information from a single sequence element. arXiv preprint arXiv:2406.18131, 2024

    arXiv 2024

  5. [5]

    P ., Matthey, L., Watters, N., Kabra, R., Higgins, I., Botvinick, M., and Lerchner, A

    Burgess, C. P ., Matthey, L., Watters, N., Kabra, R., Higgins, I., Botvinick, M., and Lerchner, A. Monet: Unsupervised scene decomposition and representation. arXiv preprint arXiv: 1901.11390, 2019

    arXiv 1901

  6. [6]

    Leveraging procedural generation to benchmark reinforcement learning

    Cobbe, K., Hesse, C., Hilton, J., and Schulman, J. Leveraging procedural generation to benchmark reinforcement learning. arXiv preprint arXiv:1912.01588, 2019

    arXiv 1912

  7. [7]

    W., Hilton, J., Klimov, O., and Schulman, J

    Cobbe, K. W., Hilton, J., Klimov, O., and Schulman, J. Phasic policy gradient. In International Conference on Machine Learning, pp. 2020–2027. PMLR, 2021

    work page 2020

  8. [8]

    Improving token-based world models with parallel observation prediction

    Cohen, L., Wang, K., Kang, B., and Mannor, S. Improving token-based world models with parallel observation prediction. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https://openreview.net/forum?id= Lfp5Dk1xb6

    work page 2024

Show all 45 references
  1. [9]

    F., Mahendran, A., van Steenkiste, S., Greff, K., Mozer, M., and Kipf, T

    Elsayed, G. F., Mahendran, A., van Steenkiste, S., Greff, K., Mozer, M., and Kipf, T. Savi++: Towards end-to-end object-centric learning from real-world videos. Neural Information Processing Systems,

  2. [10]

    R., Jones, O

    Engelcke, M., Kosiorek, A. R., Jones, O. P ., and Posner, I. Genesis: Generative scene inference and sampling with object-centric latent representations. arXiv preprint arXiv: 1907.13052, 2019

    1907 arXiv

  3. [11]

    et. al., N. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.03575, 2025

    2025 arXiv

  4. [12]

    Focus: Object-centric world models for robotics manipulation

    Ferraro, S., Mazzaglia, P ., Verbelen, T., and Dhoedt, B. Focus: Object-centric world models for robotics manipulation. arXiv preprint arXiv: 2307.02427, 2023

    2023 arXiv

  5. [13]

    and Lempitsky, V

    Ganin, Y. and Lempitsky, V . Unsupervised domain adaptation by backpropagation. InInternational conference on machine learning, pp. 1180–1189. PMLR, 2015

    2015

  6. [14]

    and Dao, T

    Gu, A. and Dao, T. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv: 2312.00752, 2023

    2023 arXiv

  7. [15]

    and Schmidhuber, J

    Ha, D. and Schmidhuber, J. Recurrent world models facilitate policy evolution. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.),Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URLhttp...

    2018

  8. [16]

    Dream to control: Learning behaviors by latent imagination

    Hafner, D., Lillicrap, T., Ba, J., and Norouzi, M. Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv: 1912.01603, 2019

    1912 arXiv

  9. [17]

    Mastering atari with discrete world models

    Hafner, D., Lillicrap, T., Norouzi, M., and Ba, J. Mastering atari with discrete world models. International Conference on Learning Representations, 2020. 10 Dyn-O: An Object-Centric World Model

    2020

  10. [18]

    Mastering diverse domains through world models

    Hafner, D., Pasukonis, J., Ba, J., and Lillicrap, T. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023

    2023 arXiv

  11. [19]

    Mastering diverse control tasks through world models

    Hafner, D., Pasukonis, J., Ba, J., and Lillicrap, T. Mastering diverse control tasks through world models. Nature, 640(8059):647–653, April 2025

    2025

  12. [20]

    Td-mpc2: Scalable, robust world models for continuous control

    Hansen, N., Su, H., and Wang, X. Td-mpc2: Scalable, robust world models for continuous control. arXiv preprint arXiv:2310.16828, 2023

    2023 arXiv

  13. [21]

    Object-centric slot diffusion

    Jiang, J., Deng, F., Singh, G., and Ahn, S. Object-centric slot diffusion. Neural Information Processing Systems, 2023. doi: 10.48550/arXiv.2303.10834

    2023 doi arXiv

  14. [22]

    Slot state space models

    Jiang, J., Deng, F., Singh, G., Lee, M., and Ahn, S. Slot state space models. arXiv preprint arXiv: 2406.12272, 2024

    2024 arXiv

  15. [23]

    F., Mahendran, A., Stone, A., Sabour, S., Heigold, G., Jonschkowski, R., Dosovitskiy, A., and Greff, K

    Kipf, T., Elsayed, G. F., Mahendran, A., Stone, A., Sabour, S., Heigold, G., Jonschkowski, R., Dosovitskiy, A., and Greff, K. Conditional object-centric learning from video. arXiv preprint arXiv:2111.12594, 2021

    2021 arXiv

  16. [24]

    Conflict-averse gradient descent for multi-task learning

    Liu, B., Liu, X., Jin, X., Stone, P ., and Liu, Q. Conflict-averse gradient descent for multi-task learning. Advances in Neural Information Processing Systems, 34, 2021

    2021

  17. [25]

    Challenging common assumptions in the unsupervised learning of disentangled representations

    Locatello, F., Bauer, S., Lucic, M., Raetsch, G., Gelly, S., Schölkopf, B., and Bachem, O. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference on machine learning, pp. 4114–4124. PMLR, 2019

    2019

  18. [26]

    Object-centric learning with slot attention, 2020

    Locatello, F., Weissenborn, D., Unterthiner, T., Mahendran, A., Heigold, G., Uszkoreit, J., Dosovitskiy, A., and Kipf, T. Object-centric learning with slot attention, 2020

    2020

  19. [27]

    Transformers are sample-efficient world models.International Conference on Learning Representations, 2022

    Micheli, V ., Alonso, E., and Fleuret, F. Transformers are sample-efficient world models.International Conference on Learning Representations, 2022. doi: 10.48550/arXiv.2209.00588

    2022 doi arXiv

  20. [28]

    Efficient world models with context-aware tokenization

    Micheli, V ., Alonso, E., and Fleuret, F. Efficient world models with context-aware tokenization. In Forty-first International Conference on Machine Learning , 2024. URL https://openreview.net/ forum?id=BiWIERWBFX

    2024

  21. [29]

    W., Schölkopf, B., Buhmann, J

    Miladinovi´ c, Ð., Gondal, M. W., Schölkopf, B., Buhmann, J. M., and Bauer, S. Disentangled state space representations. arXiv preprint arXiv:1906.03255, 2019

    1906 arXiv

  22. [30]

    Interaction-based disentanglement of entities for object- centric world models

    Nakano, A., Suzuki, M., and Matsuo, Y. Interaction-based disentanglement of entities for object- centric world models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=JQc2VowqCzz

    2023

  23. [31]

    Oord, A. v. d., Li, Y., and Vinyals, O. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  24. [32]

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H. V ., Szafraniec, M., Khalidov, V ., Fernandez, P ., Haziza, D., Massa, F., El-Nouby, A., Howes, R., Huang, P .-Y., Xu, H., Sharma, V ., Li, S.-W., Galuba, W., Rabbat, M., Assran, M., Ballas, N., Synnaeve, G., Misra, I., Jegou, H., ...

    2023

  25. [33]

    V ., Carion, N., Wu, C.-Y., Girshick, R., Dollár, P ., and Feichtenhofer, C

    Ravi, N., Gabeur, V ., Hu, Y.-T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K. V ., Carion, N., Wu, C.-Y., Girshick, R., Dollár, P ., and Feichtenhofer, C. Sam 2: Segment anything in images and videos. arXiv prepr...

    2024 arXiv

  26. [34]

    J., and Chaudhuri, S

    Sehgal, A., Grayeli, A., Sun, J. J., and Chaudhuri, S. Neurosymbolic grounding for compositional world models. International Conference on Learning Representations, 2023. doi: 10.48550/arXiv.2310. 12690

    2023 doi

  27. [35]

    Bridging the gap to real-world object-centric learning

    Seitzer, M., Horn, M., Zadaianchuk, A., Zietlow, D., Xiao, T., Simon-Gabriel, C.-J., He, T., Zhang, Z., Schölkopf, B., Brox, T., and Locatello, F. Bridging the gap to real-world object-centric learning. arXiv preprint arXiv: 2209.14860, 2022. 11 Dyn-O: An Object-Centric World Model

    2022 arXiv

  28. [36]

    C., Frossard, P ., and Vleeschouwer, C

    Simon, M. C., Frossard, P ., and Vleeschouwer, C. D. Sequential representation learning via static- dynamic conditional disentanglement. In European Conference on Computer Vision , pp. 110–126. Springer, 2024

    2024

  29. [37]

    Sutton, R. S. Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Machine learning proceedings 1990, pp. 216–224. Elsevier, 1990

    1990

  30. [38]

    Regularing generative adversarial networks under limited data

    Tseng, H.-Y., Jiang, L., Liu, C., Yang, M.-H., and Yang, W. Regularing generative adversarial networks under limited data. In CVPR, 2021

    2021

  31. [39]

    Pre-training contextualized world models with in-the-wild videos for reinforcement learning

    Wu, J., Ma, H., Deng, C., and Long, M. Pre-training contextualized world models with in-the-wild videos for reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024

    2024

  32. [40]

    Slotformer: Unsupervised visual dynamics simulation with object-centric models

    Wu, Z., Dvornik, N., Greff, K., Kipf, T., and Garg, A. Slotformer: Unsupervised visual dynamics simulation with object-centric models. International Conference on Learning Representations, 2022. doi: 10.48550/arXiv.2210.05861

    2022 doi arXiv

  33. [41]

    Slotdiffusion: Object-centric generative modeling with diffusion models

    Wu, Z., Hu, J., Lu, W., Gilitschenski, I., and Garg, A. Slotdiffusion: Object-centric generative modeling with diffusion models. Advances in Neural Information Processing Systems, 36:50932–50958, 2023

    2023

  34. [42]

    and Mandt, S

    Yingzhen, L. and Mandt, S. Disentangled sequential autoencoder. In International Conference on Machine Learning, pp. 5670–5679. PMLR, 2018

    2018

  35. [43]

    Zhao, L., Kong, L., Walters, R., and Wong, L. L. Toward compositional generalization in object- oriented world modeling. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.), Proceedings of the 39th International Conference on Machine Learnin...

    2022

  36. [44]

    Robodreamer: Learning compositional world models for robot imagination

    Zhou, S., Du, Y., Chen, J., Li, Y., Yeung, D., and Gan, C. Robodreamer: Learning compositional world models for robot imagination. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https://openr...

    2024

  37. [2022]

    doi: 10.48550/arXiv.2206.07764

    doi arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.