Pith. sign in

REVIEW 2 major objections 3 minor 1 cited by

MemoBench shows that no video world model reliably remembers an object once it leaves view; the best score still misses much of the reappearance, and models that barely move the camera inflate their apparent consistency.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 09:55 UTC pith:FMUZ4NF3

load-bearing objection MemoBench is a genuinely new diagnostic benchmark for object permanence in video world models; its ORS headline is more about re-detection than state recovery, but the broader protocol and conclusions hold up. the 2 major comments →

arxiv 2606.27537 v6 pith:FMUZ4NF3 submitted 2026-06-25 cs.CV

MemoBench: Benchmarking World Modeling in Dynamically Changing Environments

Haoyu Chen , Kaichen Zhou , Hang Hua , Kaile Zhang , Jingwen Qian , Wufei Ma , Haonan Chen , Chunjiang Liu
show 6 more authors
Yizhou Zhao Xiaoyuan Wang Weiyue Li Alan Yuille Paul Pu Liang Yilun Du
This is my paper
classification cs.CV
keywords world generationvideo generationobject permanencememory consistencydisappear-and-reappearbenchmarkworld modelsocclusion reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

MemoBench tests whether video generation models can do what toddlers eventually learn: keep track of an object after it disappears from sight and correctly recover it when it reappears. The paper creates 360 ground-truth clips, half synthetic and half real, in which a target object undergoes a physical change, leaves the field of view while that change continues, and then must reappear in its updated state. Across ten current world-generation models, none exceeds an Object Reappearance Score of 0.582, meaning no model reliably redetects the target throughout the reappearance phase. The paper also shows that models without camera conditioning can artificially top consistency metrics by producing nearly static video, so memory performance must be judged together with how much the camera actually moves. The result matters because autonomous driving, robotics, and embodied AI all require a world model to reason about state that is temporarily out of view.

Core claim

The paper's central claim is that current world-generation models lack a persistent internal representation of objects that leave the field of view. Using a Visible–Disappear–Reappear protocol, MemoBench forces a target object out of view while its physical process continues and then asks the model to regenerate it in its evolved state. The strongest result is that no evaluated model exceeds an Object Reappearance Score of 0.582, and the leading camera-conditioned model still fails to recover the target's updated state upon reappearance. A second key finding is that models without camera conditioning (and sometimes one with it) inflate consistency-oriented metrics by generating near-static v

What carries the argument

The central device is the disappear-and-reappear clip, divided into three phases by human-annotated keyframes: Visible, Disappeared, and Reappear. The load-bearing metric is the Object Reappearance Score (ORS), defined as the fraction of reappearance frames in which a text-prompted segmentation model detects the target, multiplied by the mean detection confidence across those frames. This detection-based design avoids pixel-level mask alignment when the camera viewpoint at reappearance differs from the original view. The benchmark also pairs ORS with camera-controllability scoring and motion-gated evaluation, so that a model cannot earn memory credit simply by holding the camera still.

Load-bearing premise

The Object Reappearance Score assumes that a text-prompted detector, queried with the object's original description, fairly measures reappearance success even when the physical process changes the object's appearance; a model that correctly shows the dissolved or melted state may score low simply because the prompt no longer matches what it sees.

What would settle it

Re-run the ORS evaluation on the same generated videos but prompt the detector with the expected post-occlusion appearance (e.g., 'molten wax' instead of 'wax candle'). If these re-prompted scores rise to near-perfect while original-prompt scores stay low, the paper's headline claim that all models lack object memory would be overstated.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the paper is right, video quality and consistency metrics alone cannot certify a model as a world model; occlusion memory must be tested explicitly.
  • Camera conditioning alone, even with explicit 3D representations, does not induce object permanence; memory needs its own training objective.
  • Consistency scores from near-static video are a false signal, so evaluations should always report camera controllability alongside them.
  • Phase-level analysis matters: aggregate fidelity hides the fact that all evaluated models degrade sharply when a previously seen object reappears.
  • Future world models will need a persistent state layer that updates hidden objects and is read back out when they re-enter view.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The ORS detector is prompted with the object's initial appearance, so a model that correctly renders a melted or dissolved state may be scored low; re-prompting with the expected post-occlusion state would separate detector mismatch from genuine memory failure.
  • The 'static camera cheat' suggests a natural correction for future leaderboards: report memory scores normalized by how much viewpoint change the model actually executed.
  • The consistent R-phase fidelity drop implies a testable architectural fix: feed the model an explicit state map or scene-graph memory that updates during occlusion, then measure whether ORS rises above the current ceiling.
  • If applied as a go/no-go test for embodied deployment, a reasonable threshold (e.g., motion-gated ORS above 0.9) would exclude every current model, clarifying how far the field is from usable world simulation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. MemoBench introduces a benchmark for world-model memory under the disappear-and-reappear paradigm: a target object undergoes a physical process, leaves the field of view, and must be recovered in its updated state when the camera returns. The dataset contains 360 synthetic and real-world clips with annotated visible/disappeared/reappeared phase boundaries, camera trajectories, and depth maps. The evaluation combines automated metrics (e.g., Object Reappearance Score, pixel-level fidelity, camera controllability) with a filtered LLM-based VQA suite across four dimensions. Ten video-generation models are evaluated. The main findings are that no model exceeds an ORS of about 0.58, that camera-inactive models such as LTX-Video inflate consistency-oriented scores by generating near-static video, and that camera conditioning does not by itself yield reliable object memory.

Significance. The benchmark is well constructed in several respects: the three-phase V-D-R decomposition is natural and useful; the authors provide careful keyframe annotation, metric sensitivity analyses (Table S7), motion-gated evaluation (Table S5), pose validation (Table S8), and a failure taxonomy (Table S10). The release of dataset, code, and leaderboard is a genuine contribution, and the paper correctly identifies camera inactivity as a confound for standard consistency metrics. However, the central inference that current models "lack a persistent internal representation of disappeared objects" and "fail to recover the target's updated state" rests on the ORS metric, which is a text-prompted detection score rather than a state-recovery measure. That gap is load-bearing and needs to be addressed before the headline conclusions can be accepted.

major comments (2)
  1. [§3.4, Eq. (6); §4.1; §5] ORS is defined as a SAM-3 text-prompted detection score: SAM-3 is prompted with the initially visible target description and reports a confidence if a matching mask is found in the Reappear phase. This measures re-detection of the initial appearance, not recovery of the updated state. Consequently, (a) a model that performs a 'State reset' (a failure type listed in Table S10) would receive a high SAM-3 confidence because the prompt matches the reset appearance; and (b) a model that correctly renders a dissolved, melted, or color-changed target may receive a low confidence because the prompt no longer matches. Table S4 probes prompt phrasing but does not test updated-state prompts, and no stratification by state-change magnitude is provided. The headline claim 'no model exceeds 0.582 ORS' therefore does not by itself establish that current models fail to recover the target's updated state
  2. [§3.5, Fig. 4] The VQA-based evaluation is scored by the same LLM (Gemini-3.1-Pro) that performs ground-truth filtering and failure filtering, and the reported human validation (Fig. 4) is conducted only on ground-truth videos, not on generated outputs. The conclusion that 'Continuity of Memory remains a major bottleneck' therefore rests on LLM judgments that have not been validated on the artifact-rich, state-reset, and hallucination-laden videos the benchmark is designed to distinguish. Please collect human ratings on a stratified sample of generated videos across multiple models and report human--LLM agreement on those clips, not only on ground-truth clips.
minor comments (3)
  1. [Abstract vs. §4] The abstract says 'eight state-of-the-art models' while the body and Table 2 evaluate ten models. Please make the numbers consistent.
  2. [Table S3] The caption uses 'Object Revisit Score' while the metric is defined elsewhere as 'Object Reappearance Score' (ORS). Harmonize terminology.
  3. [§4.1, Table 2 and Table S5] The camera-inactivity discussion is well supported by Table S5, but the main text's 'no model exceeds 0.6 ORS' statement uses aggregate un-gated ORS. Since the paper explicitly argues that camera inactivity inflates consistency scores, consider reporting the motion-gated ORS in the main table or prominently alongside the aggregate values.

Circularity Check

0 steps flagged

No significant circularity: MemoBench is an externally grounded measurement benchmark whose conclusions are empirical readings of defined metrics, not derivations from their own inputs.

full rationale

MemoBench is a measurement benchmark rather than a derivation chain, and its conclusions are empirical readings of externally defined metrics. The automated scores (Eqs. 2-7) are built from independent off-the-shelf tools—LAION aesthetic predictor, CLIP-IQA+, RAFT-Large, DINOv2, Depth Anything V2, SAM-3, MapAnything, ImageReward—with fixed normalization ranges; no parameter is fit to the scored model outputs and then relabeled as a prediction. The VQA pipeline is validated by GT/failure filtering and a human correlation study (92.9% agreement, Cohen's kappa = 0.85) before being applied to test videos, so the LLM judge's scores are not the same as the question-generation inputs. The only citations with author overlap ([35], [86]) are contextual/implementation support—motivating object permanence and supporting MapAnything's scalability—and neither is load-bearing for the benchmark's central findings. The ORS metric (Eq. 6) detects reappearance via SAM-3 text-prompted segmentation; inferring 'failure to recover the updated state' from low ORS is an interpretive or construct-validity step (ORS does not directly compare the updated physical state), but it is not a circular reduction because ORS is not defined in terms of 'updated state' and the paper separately triangulates with Continuity-of-Memory VQA and per-phase R-phase fidelity drops. Thus no circular step is present, and the appropriate score is 0.

Axiom & Free-Parameter Ledger

8 free parameters · 5 axioms · 0 invented entities

The benchmark's central numbers depend mainly on hand-chosen metric hyperparameters and on the validity of the detection/LLM tools used as judges; there is no fitted model parameter inside the paper itself.

free parameters (8)
  • Motion Smoothness temperature τ = 0.15
    Eq. (3); hand-chosen warp-error temperature; sensitivity analysis shows rank-stable across τ ∈ [0.05, 0.30].
  • Foreground patch fraction k in Object Identity Consistency = 40 (top-k%)
    Eq. (4); selects persistent foreground patches; Kendall's τ ≥ 0.910 across 0.2–0.8 variants.
  • Per-phase aggregation weight α = 0.7
    Eqs. (4) and (5); hand-chosen balance between mean and minimum similarities.
  • Camera Controllability stability offset θ0 = 10°
    Eq. (7); prevents division by near-zero net rotation; hand-chosen.
  • ORS coverage filtering thresholds = 0.05%–50% (fallback 0.05%–70%)
    Defines valid SAM-3 mask area; swept in Table S3 with ORS variation ≤ 0.020.
  • SAM-3 prompt template = Original subject phrase
    ORS depends strongly on prompt style (Table S4): 'detect subject in scene' drops LingBot ORS from 0.462 to 0.284.
  • VQA question budget = 24 questions/clip (6 per dimension)
    Feeds through GT/failure filtering, leaving fewer questions; per-dimension pass rates depend on this budget.
  • Normalization ranges [a, b] = Aesthetic [1,10]; CLIP-IQA+ [0,1]; ImageReward [0,1] after sigmoid
    Eq. (1); fixed ranges chosen by hand to map raw scores to 0–100.
axioms (5)
  • domain assumption Physical processes continue correctly during the disappearance interval in ground-truth clips; the human-annotated keyframes (d_start, r_start) define the true V/D/R phases.
    The entire evaluation divides clips into phases from these annotations and assumes the GT state evolution is the correct answer (Sec. 3.2).
  • domain assumption Linear interpolation of GT frames to generated-video length preserves phase alignment across frame rates, resolutions, and motion speeds.
    Sec. 3.3 downsamples GT by linearly interpolating indices; mismatched trajectories (e.g., near-static LTX-Video) are compared against GT phases.
  • domain assumption SAM-3 can detect the target object in its post-occlusion, possibly transformed state using a text description of the initial state.
    Eq. (6) and Table S4; this is the weakest assumption—correctly updated appearances may not match the prompt.
  • domain assumption The LLM judge (Gemini-3.1-Pro) answers are valid proxies for human judgment after GT/failure filtering.
    Sec. 3.5; human agreement measured on 96 questions from 12 scenes; proprietary model version may change.
  • domain assumption External tool models (RAFT, DINOv2, Depth Anything V2, MapAnything) provide sufficiently accurate signals for the metrics that use them.
    Tools are used as fixed pre-trained predictors; pose validation is reported for synthetic clips only (Table S8).

pith-pipeline@v1.3.0-alltime-deepseek · 24092 in / 16398 out tokens · 157348 ms · 2026-08-02T09:55:40.454826+00:00 · methodology

0 comments
read the original abstract

Video generation models aspire to simulate dynamic environments, and several benchmarks now evaluate memory consistency across frames. However, most assess consistency only while the target remains in view, and the few that force objects out of view evaluate static scenes where nothing changes during occlusion. To bridge this gap, we introduce MemoBench, a diagnostic benchmark built around the disappear-and-reappear paradigm in dynamically changing environments: a target object undergoes a physical process, disappears from view, and must be correctly recovered in its updated state upon reappearance. We curate 360 ground-truth clips spanning synthetic and real-world scenes, and design an evaluation suite combining automated metrics with VQA-based assessment across four diagnostic pillars. Evaluation of eight state-of-the-art models reveals key insights and open challenges regarding memory consistency under the disappear-and-reappear paradigm.

Figures

Figures reproduced from arXiv: 2606.27537 by Alan Yuille, Chunjiang Liu, Hang Hua, Haonan Chen, Haoyu Chen, Jingwen Qian, Kaichen Zhou, Kaile Zhang, Paul Pu Liang, Weiyue Li, Wufei Ma, Xiaoyuan Wang, Yilun Du, Yizhou Zhao.

Figure 1
Figure 1. Figure 1: Overview of MemoBench. Rows 1–2 show a synthetic Visible–Disappear– Reappear sequence and its camera trajectory; Rows 3–4 show a real-world state-change sequence (powder pouring). MemoBench contains 196 synthetic and 164 real-world clips, evaluated with automated metrics and LLM-judged VQA. video generation: as the virtual camera moves, objects inevitably leave and re￾enter the field of view, and the gener… view at source ↗
Figure 2
Figure 2. Figure 2: Data curation pipeline for MemoBench. Left: synthetic data (196 clips, 14 scene subdomains across 5 environment categories) generated in Unreal Engine 5. Right: real-world data (164 clips, 30 physical-state-change processes across 7 categories) captured in controlled indoor settings. estimated from the recorded RGB frames using MapAnything [29], followed by trajectory smoothing to obtain clean per-frame ca… view at source ↗
Figure 3
Figure 3. Figure 3: VQA evaluation pipeline. An LLM generates 24 polarity-balanced Yes/No questions (6 per dimension) from the prompt and first frame. Questions are filtered through ground-truth and failure-clip evaluation, then validated by human reviewers. The final question bank is applied to each generated video, producing per-dimension pass rates across four diagnostic dimensions. the refined question bank together with … view at source ↗
Figure 4
Figure 4. Figure 4: Human–VLM agreement on ground-truth videos. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison of camera controllability on a real-world clip. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Camera inactivity vs. active trajectory following. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative comparison of geometric fidelity and perceptual quality [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗

Review history (6 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Persistent Computational State: A Session-Centric Runtime for Generative World Models

    cs.AI 2026-07 conditional novelty 6.0

    World-model persistence failures are partly a serving-layer bug: restoring the runtime's non-recomputable state (observation+RNG, memory bank, or KV context) after an excursion reproduces the never-left continuation b...

Reference graph

Works this paper leans on

88 extracted references · 40 linked inside Pith · cited by 1 Pith paper

  1. [1]

    arXiv preprint arXiv:2406.03520 (2024) 2, 3

    Bansal, H., Lin, Z., Xie, T., Zong, Z., Yarom, M., Bitton, Y., Jiang, C., Sun, Y., Chang, K.W., Grover, A.: Videophy: Evaluating physical commonsense for video generation. arXiv preprint arXiv:2406.03520 (2024) 2, 3

    Pith/arXiv arXiv 2024

  2. [2]

    arXiv preprint arXiv:2503.06800 (2025) 3, 4

    Bansal, H., Peng, C., Bitton, Y., Goldenberg, R., Grover, A., Chang, K.W.: Videophy-2: A challenging action-centric physical commonsense evaluation in video generation. arXiv preprint arXiv:2503.06800 (2025) 3, 4

    Pith/arXiv arXiv 2025

  3. [3]

    arXiv preprint arXiv:2511.16719 (2025) 7

    Carion, N., Gustafson, L., Hu, Y.T., Debnath, S., Hu, R., Suris, D., Ryali, C., Alwala,K.V.,Khedr,H.,Huang,A.,etal.:Sam3:Segmentanythingwithconcepts. arXiv preprint arXiv:2511.16719 (2025) 7

    Pith/arXiv arXiv 2025

  4. [4]

    arXiv preprint arXiv:2310.19512 (2023) 3

    Chen, H., Xia, M., He, Y., Zhang, Y., Cun, X., Yang, S., Xing, J., Liu, Y., Chen, Q., Wang, X., et al.: Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512 (2023) 3

    Pith/arXiv arXiv 2023

  5. [5]

    In: CVPR (2024) 3

    Chen, H., Zhang, Y., Cun, X., Xia, M., Wang, X., Weng, C., Shan, Y.: Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In: CVPR (2024) 3

    2024

  6. [6]

    arXiv preprint arXiv:2304.14404 (2023) 3

    Chen, T.S., Lin, C.H., Tseng, H.Y., Lin, T.Y., Yang, M.H.: Motion-conditioned diffusion model for controllable video synthesis. arXiv preprint arXiv:2304.14404 (2023) 3

    Pith/arXiv arXiv 2023

  7. [7]

    In: ICLR (2023) 3

    Chen, X., Wang, Y., Zhang, L., Zhuang, S., Ma, X., Yu, J., Wang, Y., Lin, D., Qiao, Y., Liu, Z.: Seine: Short-to-long video diffusion model for generative transition and prediction. In: ICLR (2023) 3

    2023

  8. [8]

    arXiv preprint arXiv:2509.21657 (2025) 3, 10, 11, 14, S2, S3

    Dai, Y., Jiang, F., Wang, C., Xu, M., Qi, Y.: Fantasyworld: Geometry- consistent world modeling via unified video and 3d prediction. arXiv preprint arXiv:2509.21657 (2025) 3, 10, 11, 14, S2, S3

    arXiv 2025

  9. [9]

    arXiv preprint arXiv:2504.00983 (2025) 2, 3, 4

    Duan, H., Yu, H.X., Chen, S., Fei-Fei, L., Wu, J.: Worldscore: A unified evaluation benchmark for world generation. arXiv preprint arXiv:2504.00983 (2025) 2, 3, 4

    arXiv 2025

  10. [10]

    Feng, Y., Li, Y., Liu, C., Chen, Y., Jiang, F., Huang, Y., Hua, H., Yuan, Z., Zheng, K., Niu, L., et al.: Visual aesthetic benchmark: Can frontier models judge beauty? arXiv preprint arXiv:2605.12684 (2026) 8

    Pith/arXiv arXiv 2026

  11. [11]

    In: CVPR (2026) 3

    Ge, X., Pan, Y., Zhang, Y., Li, X., Zhang, W., Zhang, D., Wan, Z., Lin, X., Zhang, X., Liang, J., et al.: Airsim360: A panoramic simulation platform within drone view. In: CVPR (2026) 3

    2026

  12. [12]

    Google: Gemini 3.1 pro (2026),https://blog.google/innovation- and- ai/ models- and- research/gemini- models/gemini- 3- 1- pro/, accessed: 2026-03-02 8

    2026

  13. [13]

    arXiv preprint arXiv:1803.10122 (2018) 1, 3

    Ha, D., Schmidhuber, J.: World models. arXiv preprint arXiv:1803.10122 (2018) 1, 3

    Pith/arXiv arXiv 2018

  14. [14]

    arXiv preprint arXiv:2501.00103 (2024) 3, 10, 11, 14, S2, S3

    HaCohen, Y., Chiprut, N., Brazowski, B., Shalem, D., Moshe, D., Richardson, E., Levin, E., Shiran, G., Zabari, N., Gordon, O., et al.: Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.00103 (2024) 3, 10, 11, 14, S2, S3

    Pith/arXiv arXiv 2024

  15. [15]

    arXiv preprint arXiv:2301.04104 (2023) 1, 3

    Hafner, D., Pasukonis, J., Ba, J., Lillicrap, T.: Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104 (2023) 1, 3

    Pith/arXiv arXiv 2023

  16. [16]

    He, H., Xu, Y., Guo, Y., Wetzstein, G., Dai, B., Li, H., Yang, C.: Cameractrl: En- ablingcameracontrolfortext-to-videogeneration.arXivpreprintarXiv:2404.02101 (2024) 3

    Pith/arXiv arXiv 2024

  17. [17]

    arXiv preprint arXiv:2508.13009 (2025) 10, 11, 14, S2, S3 MemoBench 17

    He, X., Peng, C., Liu, Z., Wang, B., Zhang, Y., Cui, Q., Kang, F., Jiang, B., An, M., Ren, Y., et al.: Matrix-game 2.0: An open-source real-time and streaming interactive world model. arXiv preprint arXiv:2508.13009 (2025) 10, 11, 14, S2, S3 MemoBench 17

    Pith/arXiv arXiv 2025

  18. [18]

    arXiv preprint arXiv:2211.13221 (2022) 3

    He, Y., Yang, T., Zhang, Y., Shan, Y., Chen, Q.: Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221 (2022) 3

    Pith/arXiv arXiv 2022

  19. [19]

    Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., Fleet, D.J.: Video diffusion models. Adv. Neural Inform. Process. Syst. (2022) 3

    2022

  20. [20]

    Hore, A., Ziou, D.: Image quality metrics: Psnr vs. ssim. In: ICPR (2010) 8

    2010

  21. [21]

    In: CVPR (2023) 8

    Hu, Y., Hua, H., Yang, Z., Shi, W., Smith, N.A., Luo, J.: Promptcap: Prompt- guided image captioning for vqa with gpt-3. In: CVPR (2023) 8

    2023

  22. [22]

    In: CVPR (2023) 8

    Hu, Y., Liu, B., Kasai, J., Wang, Y., Ostendorf, M., Krishna, R., Smith, N.A.: Tifa: Accurate and interpretable text-to-image faithfulness evaluation with ques- tion answering. In: CVPR (2023) 8

    2023

  23. [23]

    In: ECCV (2024) 8

    Hua, H., Shi, J., Kafle, K., Jenni, S., Zhang, D., Collomosse, J., Cohen, S., Luo, J.: Finematch: Aspect-based fine-grained image and text mismatch detection and correction. In: ECCV (2024) 8

    2024

  24. [24]

    arXiv preprint arXiv:2410.09733 (2024) 8

    Hua, H., Tang, Y., Zeng, Z., Cao, L., Yang, Z., He, H., Xu, C., Luo, J.: Mmcompo- sition: Revisiting the compositionality of pre-trained vision-language models. arXiv preprint arXiv:2410.09733 (2024) 8

    Pith/arXiv arXiv 2024

  25. [25]

    arXiv preprint arXiv:2505.19415 (2025) 8

    Hua, H., Zeng, Z., Song, Y., Tang, Y., He, L., Aliaga, D., Xiong, W., Luo, J.: Mmig-bench: Towards comprehensive and explainable evaluation of multi-modal image generation models. arXiv preprint arXiv:2505.19415 (2025) 8

    Pith/arXiv arXiv 2025

  26. [26]

    In: CVPR (2024) 2, 3, 4, 6

    Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., et al.: Vbench: Comprehensive benchmark suite for video genera- tive models. In: CVPR (2024) 2, 3, 4, 6

    2024

  27. [27]

    arXiv preprint (2025) 3, 10, 11, 14, S2, S3

    HunyuanWorld, T.: Hy-world 1.5: A systematic framework for interactive world modeling with real-time latency and geometric consistency. arXiv preprint (2025) 3, 10, 11, 14, S2, S3

    2025

  28. [28]

    arXiv preprint arXiv:2411.02385 (2024) 3

    Kang, B., Yue, Y., Lu, R., Lin, Z., Zhao, Y., Wang, K., Huang, G., Feng, J.: How far is video generation from world model: A physical law perspective. arXiv preprint arXiv:2411.02385 (2024) 3

    Pith/arXiv arXiv 2024

  29. [29]

    arXiv preprint arXiv:2509.13414 (2025) 5, 8

    Keetha, N., Müller, N., Schönberger, J., Porzi, L., Zhang, Y., Fischer, T., Knapitsch, A., Zauss, D., Weber, E., Antunes, N., et al.: Mapanything: Univer- sal feed-forward metric 3d reconstruction. arXiv preprint arXiv:2509.13414 (2025) 5, 8

    Pith/arXiv arXiv 2025

  30. [30]

    Kuaishou: Kling (2024),https://kling.kuaishou.com/en, accessed: 2026-03-01 3

    2024

  31. [31]

    arXiv preprint arXiv:2502.20694 (2025) 2, 3, 4

    Li, D., Fang, Y., Chen, Y., Yang, S., Cao, S., Wong, J., Luo, M., Wang, X., Yin, H., Gonzalez, J.E., et al.: Worldmodelbench: Judging video generation models as world models. arXiv preprint arXiv:2502.20694 (2025) 2, 3, 4

    Pith/arXiv arXiv 2025

  32. [32]

    arXiv preprint arXiv:2506.17201 (2025) 3, 10, 11, 14, S2, S3

    Li, J., Tang, J., Xu, Z., Wu, L., Zhou, Y., Shao, S., Yu, T., Cao, Z., Lu, Q.: Hunyuan-gamecraft: High-dynamic interactive game video generation with hybrid history condition. arXiv preprint arXiv:2506.17201 (2025) 3, 10, 11, 14, S2, S3

    Pith/arXiv arXiv 2025

  33. [33]

    arXiv preprint arXiv:2601.03444 (2026) 8

    Li, W., Zhao, M., Dong, W., Cai, J., Wei, Y., Pocress, M., Li, Y., Yuan, W., Wang, X., Hou, R., et al.: Grading scale impact on llm-as-a-judge: Human-llm alignment is highest on 0-5 grading scale. arXiv preprint arXiv:2601.03444 (2026) 8

    arXiv 2026

  34. [34]

    Authorea Preprints (2026) 3

    Li, Y., Meng, S., Yang, C., Feng, W., Liu, J., An, Z., Wang, Y., Tian, Y.: A comprehensive survey of interaction techniques in 3d scene generation. Authorea Preprints (2026) 3

    2026

  35. [35]

    arXiv preprint arXiv:2601.01075 (2026) 2 18 H

    Lillemark, H.J., Huang, B., Zhan, F., Du, Y., Keller, T.A.: Flow equivariant world models: Memory for partially observed dynamic environments. arXiv preprint arXiv:2601.01075 (2026) 2 18 H. Chen et al

    Pith/arXiv arXiv 2026

  36. [36]

    arXiv preprint arXiv:2412.00131 (2024) 3

    Lin, B., Ge, Y., Cheng, X., Li, Z., Zhu, B., Wang, S., He, X., Ye, Y., Yuan, S., Chen, L., et al.: Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.00131 (2024) 3

    Pith/arXiv arXiv 2024

  37. [37]

    In: CVPR (2026) 3

    Lin, X., Song, M., Zhang, D., Lu, W., Li, H., Du, B., Yang, M.H., Nguyen, T., Qi, L.: Depth any panoramas: A foundation model for panoramic depth estimation. In: CVPR (2026) 3

    2026

  38. [38]

    arXiv preprint arXiv:2404.01291 (2024) 8

    Lin, Z., Pathak, D., Li, B., Li, J., Xia, X., Neubig, G., Zhang, P., Ramanan, D.: Evaluating text-to-visual generation with image-to-text generation. arXiv preprint arXiv:2404.01291 (2024) 8

    Pith/arXiv arXiv 2024

  39. [39]

    arXiv preprint arXiv:2603.06022 (2026) 3

    Liu, C., Wang, X., Lin, Q., Xiao, A., Chen, H., Wen, S., Zhang, H., Qi, L., Yang, M.H., Jeni, L.A., et al.: Mosiv: Multi-object system identification from videos. arXiv preprint arXiv:2603.06022 (2026) 3

    arXiv 2026

  40. [40]

    In: Proceedings of the AAAI Conference on Artificial Intelligence (2026) 3

    Liu,M.,Liu,J.,Zhang,Y.,Li,J.,Yang,M.Y.,Nex,F.,Cheng,H.:4dstr:Advancing generative 4d gaussians with spatial-temporal rectification for high-quality and consistent 4d generation. In: Proceedings of the AAAI Conference on Artificial Intelligence (2026) 3

    2026

  41. [41]

    arXiv preprint arXiv:2604.04198 (2026) 3

    Liu, M., Zhang, D., Liu, J., Cui, J., Xie, H., Chen, G., Ye, H., Yang, M.Y., Nex, F., Cheng, H.: Driveva: Video action models are zero-shot drivers. arXiv preprint arXiv:2604.04198 (2026) 3

    Pith/arXiv arXiv 2026

  42. [42]

    arXiv preprint arXiv:2402.008271(2024) 3

    Liu, P., Song, L., Zhang, D., Hua, H., Tang, Y., Tu, H., Luo, J., Xu, C.: Emo- avatar: Efficient monocular video style avatar through texture rendering. arXiv preprint arXiv:2402.008271(2024) 3

    Pith/arXiv arXiv 2024

  43. [43]

    Liu, Y., Li, L., Ren, S., Gao, R., Li, S., Chen, S., Sun, X., Hou, L.: Fetv: A benchmark for fine-grained evaluation of open-domain text-to-video generation. Adv. Neural Inform. Process. Syst. (2023) 3

    2023

  44. [44]

    arXiv preprint arXiv:2603.30045 (2026) 3

    Liu, Y., Lin, X., Li, X., Yang, B., Wang, C., Sunkavalli, K., Hold-Geoffroy, Y., Tan, H., Zhang, K., Xie, X., et al.: Omniroam: World wandering via long-horizon panoramic video generation. arXiv preprint arXiv:2603.30045 (2026) 3

    arXiv 2026

  45. [45]

    Luma AI: Luma dream machine (2024),https://lumalabs.ai/dream-machine, accessed: 2026-03-01 3

    2024

  46. [46]

    arXiv preprint arXiv:2303.08320 (2023) 3

    Luo, Z., Chen, D., Zhang, Y., Huang, Y., Wang, L., Shen, Y., Zhao, D., Zhou, J., Tan, T.: Videofusion: Decomposed diffusion models for high-quality video genera- tion. arXiv preprint arXiv:2303.08320 (2023) 3

    Pith/arXiv arXiv 2023

  47. [47]

    In: ICCV (2025) 8

    Ma, W., Chen, H., Zhang, G., Chou, Y.C., Chen, J., de Melo, C., Yuille, A.: 3dsrbench: A comprehensive 3d spatial reasoning benchmark. In: ICCV (2025) 8

    2025

  48. [48]

    OpenAI: Sora (2024),https://openai.com/index/sora/, accessed: 2026-03-01 3

    2024

  49. [49]

    OpenAI: Sora2 (2025),https://openai.com/index/sora-2/, accessed: 2026-03-01 3

    2025

  50. [50]

    arXiv preprint arXiv:2304.07193 (2023) 6

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023) 6

    Pith/arXiv arXiv 2023

  51. [51]

    arXiv preprint arXiv:2503.09642 (2025) 1, 3, 10, 11, 14, S2, S3

    Peng, X., Zheng, Z., Shen, C., Young, T., Guo, X., Wang, B., Xu, H., Liu, H., Jiang, M., Li, W., et al.: Open-sora 2.0: Training a commercial-level video generation model in 200 k. arXiv preprint arXiv:2503.09642 (2025) 1, 3, 10, 11, 14, S2, S3

    Pith/arXiv arXiv 2025

  52. [52]

    arXiv preprint arXiv:2410.18072 (2024) 3, 4

    Qin, Y., Shi, Z., Yu, J., Wang, X., Zhou, E., Li, L., Yin, Z., Liu, X., Sheng, L., Shao, J., et al.: Worldsimbench: Towards video generation models as world simulators. arXiv preprint arXiv:2410.18072 (2024) 3, 4

    Pith/arXiv arXiv 2024

  53. [53]

    com / research / introducing-gen-3-alpha, accessed: 2026-03-01 3 MemoBench 19

    Runway ML: Gen-3 alpha (2024),https : / / runwayml . com / research / introducing-gen-3-alpha, accessed: 2026-03-01 3 MemoBench 19

    2024

  54. [54]

    Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al.: Laion-5b: An open large- scale dataset for training next generation image-text models. Adv. Neural Inform. Process. Syst. (2022) 6

    2022

  55. [55]

    arXiv preprint arXiv:2209.14792 (2022) 3

    Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., et al.: Make-a-video: Text-to-video generation without text- video data. arXiv preprint arXiv:2209.14792 (2022) 3

    Pith/arXiv arXiv 2022

  56. [56]

    In: CVPR (2025) 3

    Sun, K., Huang, K., Liu, X., Wu, Y., Xu, Z., Li, Z., Liu, X.: T2v-compbench: A comprehensive benchmark for compositional text-to-video generation. In: CVPR (2025) 3

    2025

  57. [57]

    arXiv preprint arXiv:2601.20540 (2026) 1, 3, 10, 11, 14, S2

    Team, R., Gao, Z., Wang, Q., Zeng, Y., Zhu, J., Cheng, K.L., Li, Y., Wang, H., Xu, Y., Ma, S., Chen, Y., Liu, J., Cheng, Y., Yao, Y., Zhu, J., Meng, Y., Zheng, K., Bai, Q., Chen, J., Shen, Z., Yu, Y., Zhu, X., Shen, Y., Ouyang, H.: Advancing open-source world models. arXiv preprint arXiv:2601.20540 (2026) 1, 3, 10, 11, 14, S2

    Pith/arXiv arXiv 2026

  58. [58]

    In: ECCV (2020) 6

    Teed, Z., Deng, J.: Raft: Recurrent all-pairs field transforms for optical flow. In: ECCV (2020) 6

    2020

  59. [59]

    In: ECCV (2024) 3

    Voleti, V., Yao, C.H., Boss, M., Letts, A., Pankratz, D., Tochilkin, D., Laforte, C., Rombach, R., Jampani, V.: Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion. In: ECCV (2024) 3

    2024

  60. [60]

    arXiv preprint arXiv:2503.20314 (2025) 3, 10, 11, 14, S2

    Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.W., Chen, D., Yu, F., Zhao, H., Yang, J., et al.: Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 (2025) 3, 10, 11, 14, S2

    Pith/arXiv arXiv 2025

  61. [61]

    arXiv preprint arXiv:2605.13169 (2026) 8

    Wang, C., Lin, X., Liu, J., Liu, Y., Wang, Z., Qi, D., Yan, Y., Chen, X.: PanoWorld: Towards spatial supersensing in 360-degree panorama world. arXiv preprint arXiv:2605.13169 (2026) 8

    Pith/arXiv arXiv 2026

  62. [62]

    In: AAAI (2023) 6

    Wang, J., Chan, K.C., Loy, C.C.: Exploring clip for assessing the look and feel of images. In: AAAI (2023) 6

    2023

  63. [63]

    arXiv preprint arXiv:2308.06571 (2023) 3

    Wang, J., Yuan, H., Chen, D., Zhang, Y., Wang, X., Zhang, S.: Modelscope text- to-video technical report. arXiv preprint arXiv:2308.06571 (2023) 3

    Pith/arXiv arXiv 2023

  64. [64]

    arXiv preprint arXiv:2506.19291 (2025) 3

    Wang, X., Zhao, Y., Ye, B., Shan, X., Lyu, W., Qi, L., Chan, K.C., Li, Y., Yang, M.H.: Holigs: Holistic gaussian splatting for embodied view synthesis. arXiv preprint arXiv:2506.19291 (2025) 3

    Pith/arXiv arXiv 2025

  65. [65]

    IJCV (2025) 3

    Wang, Y., Chen, X., Ma, X., Zhou, S., Huang, Z., Wang, Y., Yang, C., He, Y., Yu, J., Yang, P., et al.: Lavie: High-quality video generation with cascaded latent diffusion models. IJCV (2025) 3

    2025

  66. [66]

    IEEE TIP (2004) 8

    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE TIP (2004) 8

    2004

  67. [67]

    In: SIGGRAPH (2024) 3

    Wang,Z.,Yuan,Z.,Wang,X.,Li,Y.,Chen,T.,Xia,M.,Luo,P.,Shan,Y.:Motionc- trl: A unified and flexible motion controller for video generation. In: SIGGRAPH (2024) 3

    2024

  68. [68]

    arXiv preprint arXiv:2406.09455 (2024) 3

    Xiang, J., Liu, G., Gu, Y., Gao, Q., Ning, Y., Zha, Y., Feng, Z., Tao, T., Hao, S., Shi, Y., et al.: Pandora: Towards general world model with natural language actions and video states. arXiv preprint arXiv:2406.09455 (2024) 3

    Pith/arXiv arXiv 2024

  69. [69]

    In: ECCV (2024) 3

    Xing,J.,Xia,M.,Zhang,Y.,Chen,H.,Yu,W.,Liu,H.,Liu,G.,Wang,X.,Shan,Y., Wong, T.T.: Dynamicrafter: Animating open-domain images with video diffusion priors. In: ECCV (2024) 3

    2024

  70. [70]

    arXiv preprint arXiv:2406.02509 (2024) 3 20 H

    Xu, D., Nie, W., Liu, C., Liu, S., Kautz, J., Wang, Z., Vahdat, A.: Camco: Camera-controllable 3d-consistent image-to-video generation. arXiv preprint arXiv:2406.02509 (2024) 3 20 H. Chen et al

    Pith/arXiv arXiv 2024

  71. [71]

    Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., Dong, Y.: Imagere- ward: Learning and evaluating human preferences for text-to-image generation. Adv. Neural Inform. Process. Syst. (2023) 8

    2023

  72. [72]

    Yang, L., Kang, B., Huang, Z., Zhao, Z., Xu, X., Feng, J., Zhao, H.: Depth anything v2. Adv. Neural Inform. Process. Syst. (2024) 7

    2024

  73. [73]

    arXiv preprint arXiv:2408.06072 (2024) 3, 10, 11, 14, S2, S3

    Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., Yang, Y., Hong, W., Zhang, X., Feng, G., et al.: Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072 (2024) 3, 10, 11, 14, S2, S3

    Pith/arXiv arXiv 2024

  74. [74]

    arXiv preprint arXiv:2409.02048 (2024) 3

    Yu, W., Xing, J., Yuan, L., Hu, W., Li, X., Huang, Z., Gao, X., Wong, T.T., Shan, Y., Tian, Y.: Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409.02048 (2024) 3

    Pith/arXiv arXiv 2024

  75. [75]

    Yuan, S., Huang, J., Xu, Y., Liu, Y., Zhang, S., Shi, Y., Zhu, R.J., Cheng, X., Luo, J., Yuan, L.: Chronomagic-bench: A benchmark for metamorphic evaluation of text-to-time-lapse video generation. Adv. Neural Inform. Process. Syst. (2024) 3

    2024

  76. [76]

    IJCV (2025) 3

    Zhang, D.J., Wu, J.Z., Liu, J.W., Zhao, R., Ran, L., Gu, Y., Gao, D., Shou, M.Z.: Show-1: Marrying pixel and latent diffusion models for text-to-video generation. IJCV (2025) 3

    2025

  77. [77]

    arXiv preprint arXiv:2510.18135 (2025) 3, 4

    Zhang, J., Jiang, M., Dai, N., Lu, T., Uzunoglu, A., Zhang, S., Wei, Y., Wang, J., Patel, V.M., Liang, P.P., et al.: World-in-world: World models in a closed-loop world. arXiv preprint arXiv:2510.18135 (2025) 3, 4

    arXiv 2025

  78. [78]

    In: CVPR (2018) 8

    Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: CVPR (2018) 8

    2018

  79. [79]

    arXiv preprint arXiv:2508.01112 (2025) 3

    Zhao, Y., Chen, H., Liu, C., Li, Z., Herrmann, C., Hur, J., Li, Y., Yang, M.H., Raj, B., Xu, M.: Masiv: Toward material-agnostic system identification from videos. arXiv preprint arXiv:2508.01112 (2025) 3

    Pith/arXiv arXiv 2025

  80. [80]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2025) 3

    Zhao, Y., Liu, C., Chen, H., Raj, B., Xu, M., Baltrusaitis, T., Rundle, M., Wu, H., Ghasedi, K.: Total-editing: Head avatar with editable appearance, motion, and lighting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2025) 3

    2025

Showing first 80 references.