{"id":"78732872-0bf6-492c-92d2-3475fb2c255b","arxiv_id":"2608.06799","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PSG-JEPA grounds JEPA world-model latents in robot proprioception and joint-angle changes, improving physical state identifiability, planning, and policy learning.","lead":"This paper adds two training-only objectives to a latent world model: grounding each latent in the robot's joint state, and grounding pairs of latents in multi-horizon joint-angle changes. This makes robot state easier to read out from the latent space and improves goal-conditioned planning and policy success in simulation and on a real robot.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Q1 probes verify the training targets themselves, and no baseline controls for a matched non-physical auxiliary objective; the causal claim that physical grounding specifically drives the gains is not yet isolated.","rationale":"I read the paper as claiming that physical state grounding, not just additional auxiliary supervision, is what improves JEPA latents. The empirical pattern is internally consistent, and the planning/policy experiments are the right non-circular evidence for the downstream claim. However, the first evidence level (Q1) is partly tautological because the probe targets coincide with the training targets of the discarded heads, and the Q2/Q3 comparisons do not include a non-physical auxiliary control. The reader's weakest assumption was temporal alignment of state and camera streams; I agree that is a real implementation risk especially for the real-robot result, but the larger gap is the missing control for physical content. This does not reject the paper; it strengthens the conditions under which the central claim can be accepted. The reader's CONDITIONAL verdict should remain, with the added condition that the physical-content attribution be tested against a random-target or permuted-label control (or that the grounding pipeline be released for inspection).","tokens_in":12753,"tokens_out":12847,"duration_ms":147609,"concrete_test":"Train a random-grounding PSG-JEPA variant with the same state and transition heads, loss weights, optimizer, and training budget, but with the targets (s_t and Delta q_{t,k}) replaced by (i) fixed random Gaussian vectors with the same dimensions and (ii) time-permuted versions of the true states. Rerun the Q2 OGBench planning sweep (Table 3) and the Q3 LIBERO-Goal policy protocol (Table 4) for both variants. If either variant matches PSG-JEPA's success rates within seed noise, the physical content of the targets is not the mechanism; if both fall back toward the LeWM/LeWMActionIDM level, the physical-grounding attribution is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Static grounding (Eq. 4) and dynamic grounding (Eq. 6) are exactly MSE regressions from z_t to s_t and from (z_t, z_{t+k}) to Delta q_{t,k}. The Q1 probes in Tables 1-2 then fit linear/MLP regressors to predict the same s_t and one-step transition quantities from frozen latents. High PSG-JEPA probe r is therefore expected by construction: a low-loss solution of the training objective already provides a decoding path (H_s or H_Delta) for those targets. The comparison to LeWM/DINOv2 does not independently establish that forward-only training leaves a robot-centric identifiability gap; it shows that adding supervised targets makes those targets decodable. The load-bearing non-circular evidence is Q2/Q3, but neither LeWM nor LeWMActionIDM is a matched auxiliary-objective control for the physical content of the targets: LeWMActionIDM uses a different action target, and LeWM has no auxiliary head. It remains possible that most of the planning/policy gains come from extra regression supervision or from the auxiliary head architecture rather than from the physical meaning of proprioception and joint-angle deltas. The reader's alignment concern is real, but it is secondary here: simulation data can be exactly aligned, and the real-robot experiment is only one baseline without variance reporting.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper asks whether action-conditioned forward prediction in JEPA world models is sufficient for robot control, and argues that it is not: forward-only training leaves robot-centric physical state poorly identifiable from latents. The authors propose PSG-JEPA, which adds two training-only grounding losses to the LeWM backbone: a static loss regressing each latent to logged proprioceptive state (Eq. 4) and a dynamic loss regressing each latent pair to multi-horizon joint-angle changes (Eq. 6). The grounding heads are discarded at inference. Evaluation is at three levels: probe-based identifiability (Tables 1-2), goal-conditioned planning on frozen latents with the GC-IDM planner (Table 3, Fig. 3), and policy learning in LIBERO-Goal and on a real dual-arm robot (Table 4, Fig. 4). Across these levels, PSG-JEPA reports higher scores than LeWM, LeWMActionIDM, and DINOv2, and ablations (Table 5) indicate that both grounding terms contribute.","tokens_in":13086,"tokens_out":7301,"duration_ms":74254,"significance":"If the causal claim is accepted, the paper makes a practically useful contribution: a simple, inference-free modification to JEPA world-model training improves downstream planning and policy performance, and the evaluation is unusually thorough for this area, spanning probes, planning, simulation policy learning, and real-robot manipulation. The same-backbone comparison to LeWM and the three-seed reporting in simulation are strengths. The main caveat is that the Q1 identifiability results are partly by construction, and no baseline controls for a matched non-physical auxiliary objective, so the specifically physical nature of the benefit is not yet isolated. With that isolation, the result would be significant; as it stands, it is a promising but incompletely supported causal claim.","major_comments":[{"comment":"The Q1 probe evaluation partly re-measures the training objectives, so it does not independently establish an identifiability gap. Table 1 probes predict exactly the components of s_t used as targets in Eq. (4), and the JointVel row of Table 2 is exactly the k=1 target of Eq. (6). High probe r for PSG-JEPA therefore reflects the existence of the trained decoding heads H_s and H_Delta. The comparison to LeWM shows that supervised targets help, but it does not show that the physical content of those targets, rather than the mere addition of regression supervision or auxiliary heads, is what creates the gain. Please add a matched auxiliary-objective control (e.g., a head regressing a scrambled or non-physical target with identical architecture and loss weight) and, in the absence of such a control, soften the identifiability-gap claim.","section":"Section 4.3, Tables 1-2; Eq. (4), Eq. (6)"},{"comment":"The central causal claim that physical grounding, rather than extra regression supervision in general, drives the planning and policy gains is not isolated. LeWMActionIDM adds an action-prediction target but uses no state head and a different target, and LeWM has no auxiliary head; neither is a matched control for the architecture or for the amount of additional supervision. Please include a control with the same head architecture and training budget but a non-physical target (or a permutation or scrambling of the physical targets) to show that the physical meaning of the targets is what matters. Without this, the title's claim that forward prediction is insufficient is not yet fully supported.","section":"Section 4.4-4.5, Tables 3-4"},{"comment":"The real-robot results are reported without variance or replication: Fig. 4 gives one success rate per task from a single run of 50 trials, and no information is provided about the number of policy-training seeds or runs. Please report multiple seeds or at least trial-level confidence intervals, and describe how logged robot state is synchronized with camera frames. Figure 2 only calls the state 'aligned' without specifying the alignment procedure, the timestamp source, or the expected synchronization error, which matters because Eq. (4) and Eq. (6) pair each latent with the state at that frame.","section":"Figure 2 and Section 4.5, Real-World Robot"}],"minor_comments":[{"comment":"The x-axis label '1 5 25 100' does not match the epoch values in Table 3 (5, 10, 25, 100); please check the axis values and use consistent notation.","section":"Figure 3"},{"comment":"The open-loop prediction results are reported only as text in the paragraph without a table or figure; consider adding a small table or plot for reproducibility.","section":"Section 4.4, Open-loop latent prediction"},{"comment":"The dimension d_s of the proprioceptive state is not defined before Eq. (4); please define d_s explicitly alongside d_q.","section":"Section 3.2, Eq. (4)"},{"comment":"The reference for the OFT action head, 'Li et al. (2026), Predictive but not plannable: RC-aux for latent world models,' seems mismatched with the OFT method; please verify and correct the citation.","section":"Section 4.5, OFT policy head"},{"comment":"The 'state' and 'transition' probe rows report means of linear Pearson r across heterogeneous quantities; please state explicitly which quantities are averaged and consider reporting per-quantity values, since averaging correlations over different scales can be misleading.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a robotics or world-model venue. The central idea is simple and likely useful, but the causal isolation needs strengthening. The overlap between Q1 probes and training targets should be acknowledged in revision; it is not fatal if the authors add the requested control experiments and reframe the identifiability claim. The real-robot section also needs variance reporting before the empirical claims can be assessed quantitatively."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth knowing: this paper proposes a training-only modification to LeWM—regress latents to proprioceptive state and latent pairs to multi-horizon joint-angle changes—and reports consistent gains in probe identifiability, goal-conditioned planning, and policy learning. I think the downstream gains are probably real, but the diagnosis is weaker than the headline suggests because the main identifiability probes largely verify the training targets themselves.\n\nWhat's new: the combination is new for end-to-end JEPA world models, and the multi-horizon joint-angle change target is a sensible choice, sidestepping the action-sequence ambiguity of inverse-dynamics targets. The evaluation is thorough: three levels, ablations, simulation plus a real robot. The planning and policy results are external benchmarks, so the central contribution is supported independently of the probes.\n\nSoft spots, in order. First, the Q1 probes are close to circular. Static and dynamic grounding are MSE regressions from z_t to s_t and from (z_t, z_{t+k}) to Delta q. The probes then fit regressors to predict the same quantities from the same latents. A low-loss solution of the training objective provides a decoding path, so high probe r is expected by construction. This doesn't falsify the identifiability narrative entirely, but it means the diagnostic claim about forward-only training leaving a 'gap' is not independently established. The load-bearing evidence is Q2/Q3. Second, no baseline controls for a matched non-physical auxiliary objective. LeWMActionIDM uses action prediction, not a regression head on state-like targets with the same architecture and weight. So you can't isolate whether the gains come from the physical content of the supervision or just from extra regression supervision plus the auxiliary head. This is the main threat to the causal claim. It is addressable: add a variant that predicts, say, random or constant targets with the same heads. Third, the real-robot results have no error bars and compare only to LeWM. That is a minor issue given the simulation results are consistent. Fourth, no code or data, which limits reproducibility.\n\nThe alignment concern about state and image streams is real but secondary: simulation can be exactly aligned, and the real-robot experiment is one baseline without variance.\n\nBottom line: this is a useful incremental contribution for the latent-world-model crowd. The method is simple, training-only, and the downstream results are consistent. It deserves a serious referee; the main revision should be a matched control and a softened identifiability claim, plus code/data release.\n\nRecommendation: send to peer review, likely accept after revisions.","headline":"A solid empirical paper whose downstream gains are probably real, but whose identifiability diagnosis is partly circular and whose causal claim lacks a matched control.","tokens_in":13576,"tokens_out":2499,"would_cite":true,"duration_ms":23102,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Physical state grounding beats forward-only JEPA world models","keywords":["physical state grounding","JEPA world models","latent identifiability","goal-conditioned planning","policy learning","proprioception","joint-angle changes","robot manipulation"],"falsifier":"On OGBench-Cube, take the same PSG-JEPA training pipeline and shift the logged joint-angle stream by 2–5 control steps relative to the images, then rerun the single-latent/transition probes and the 5-epoch GC-IDM planning evaluation. If the identifiability gains and fast-planning lead over LeWM disappear or reverse, the result depends on the assumed alignment rather than on the grounding targets per se.","tokens_in":12517,"feed_emoji":"🤖","tokens_out":7807,"duration_ms":62835,"temperature":0.7,"pith_summary":"This paper argues that forward prediction alone is not enough for a JEPA world model to serve robot control, because the learned latents do not reliably expose physical state or state changes. It proposes PSG-JEPA, which adds two training-only grounding objectives: a state head that predicts robot proprioception from each latent, and a transition head that predicts multi-horizon joint-angle changes from latent pairs. In probe-based identifiability, goal-conditioned planning on frozen latents, and policy learning in simulation and on a real robot, PSG-JEPA consistently beats forward-only, action-supervised, and frozen-feature baselines. The inference architecture and computational cost remain unchanged because the grounding heads are discarded after training.","feed_headline":"Physical state grounding beats forward-only JEPA world models","feed_subtitle":"Adding proprioception and joint-angle-change targets improves identifiability, planning, and real-robot policies.","key_machinery":"The argument rides on two lightweight heads attached to the shared JEPA latent during training. The state head $H_s$ regresses each latent $z_t$ to the aligned proprioceptive state $s_t$, making physical state readable from a single latent; the transition head $H_\\Delta$ regresses every endpoint pair $(z_{t+i}, z_{t+i+k})$ to the joint-angle change $\\Delta q_{t+i,k}=q_{t+i+k}-q_{t+i}$ for all horizons $k=1,\\dots,T-1$, with each horizon weighted equally so short-range pairs do not dominate. The total objective is $L_{PSG}=L_{JEPA}+\\lambda_g(L_{static}+L_{dynamic})$, where $L_{JEPA}$ retains forward prediction and the SIGReg anti-collapse term. Because $\\Delta q$ is uniquely determined by endpoint states and fixed-dimensional across horizons, it avoids the ambiguity of inverse-dynamics action supervision.","core_discovery":"The paper's central claim is that forward-trained JEPA representations suffer a robot-centric identifiability gap: single latents fail to decode important proprioceptive variables such as end-effector yaw, and latent pairs only partially decode physical transitions. PSG-JEPA closes this gap by supervising the shared planning latent with proprioceptive state and with fixed-dimensional joint-angle changes across all horizons in the training window. As a result, the same backbone that underperforms at 5-epoch planning (LeWM, 80.7% success) reaches 95.0% when trained with grounding, and the grounded representation also lifts LIBERO-Goal policy success from 77.7% to 85.3% and real-robot average success from 60.0% to 79.3%. The grounding heads are removed at inference, so the improvement comes at zero added deployment cost.","pith_inferences":["The identifiability-gap diagnosis suggests that forward-prediction objectives are not state-anchored; any cheap, temporally aligned physical measurement (Cartesian pose, object state, contact forces) could be swapped into the same grounding recipe, a testable extension for other robot domains.","Because the grounding heads are disposable, the same forward backbone could be re-grounded for a new robot or task with a short fine-tuning pass, offering a cheap domain-adaptation route for latent world models.","The results implicitly hinge on state–image temporal alignment. A direct test would deliberately misalign the logged joint-angle stream by a few control steps and check whether the identifiability and planning gains vanish; the paper neither describes its alignment procedure nor reports such a stress test."],"forward_implications":["Planners on grounded latents need far less data and optimization: at 5 GC-IDM epochs PSG-JEPA already scores 95.0% on OGBench-Cube versus 80.7% for LeWM, and at 25% demonstration data it keeps a double-digit lead.","The same grounded representation helps task-conditioned policies, not just planning: +7.6 points on LIBERO-Goal and +19.3 points average on three real-robot tasks over the LeWM baseline.","Grounding does not degrade the forward model: recursive latent prediction error falls at every horizon, with a 67% lower MSE at the longest rollout on OGBench-Cube.","Because the grounding heads are discarded after training, PSG-JEPA is a drop-in training-time change to LeWM-style JEPA world models with no inference-time overhead.","Multi-horizon transition grounding contributes beyond adjacent-pair supervision: replacing it with adjacent-only or endpoint-only horizons drops policy success from 85.3% to about 81% and planning from 95.0% to about 93.5%."],"supporting_citations":[{"why":"Supplies the LeWM forward-prediction JEPA backbone and the primary same-backbone baseline that PSG-JEPA augments.","marker":"Maes et al. (2026)"},{"why":"Supplies the SIGReg anti-collapse regularizer retained in the forward JEPA objective.","marker":"Balestriero and LeCun (2025)"},{"why":"Provides OGBench-Cube and OGBench-Scene, the datasets used for probe, planning, and open-loop prediction evaluations.","marker":"Park et al. (2025)"},{"why":"Provides LIBERO-Goal, the benchmark used for the task-conditioned policy-learning comparison.","marker":"Liu et al. (2023)"},{"why":"Supplies the GC-IDM amortized goal-conditioned planner used for frozen-latent planning evaluations.","marker":"Nguyen et al. (2026)"},{"why":"Provides the Mobile ALOHA dual-arm teleoperation design used for the real-robot evaluation.","marker":"Fu et al. (2024)"},{"why":"Provides the pretrained DINOv2 features used as a frozen-feature baseline.","marker":"Oquab et al. (2023)"},{"why":"Provides V-JEPA 2, the video-JEPA baseline family that motivates the JEPA world-model setting.","marker":"Assran et al. (2025)"}],"fun_headline_variants":["Grounding JEPA latents lifts real-robot success to 79%","Zero-cost latent grounding outperforms forward prediction","Physical grounding boosts JEPA planning and policy","JEPA grounding: 95% planning, 79% real-robot success"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes the logged robot state is temporally aligned with the camera frames used to compute latents, but it never describes the alignment procedure or quantifies synchronization error; if the streams are off by even a few control steps, the grounding targets become noisy and the claimed gains could dissolve.","fun_headline_variants_meta":{"raw":{"variants":["Grounding JEPA latents lifts real-robot success to 79%","Zero-cost latent grounding outperforms forward prediction","Physical grounding boosts JEPA planning and policy","JEPA grounding: 95% planning, 79% real-robot success"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000492,"raw_usage":{"total_tokens":2400,"prompt_tokens":906,"completion_tokens":1494,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":522,"completion_tokens_details":{"reasoning_tokens":1424}},"tokens_in":522,"tokens_out":1494,"duration_ms":11402,"temperature":1.0,"reasoning_tokens":1424,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:35:08.876161+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On OGBench-Cube, take the same PSG-JEPA training pipeline and shift the logged joint-angle stream by 2–5 control steps relative to the images, then rerun the single-latent/transition probes and the 5-epoch GC-IDM planning evaluation. If the identifiability gains and fast-planning lead over LeWM disappear or reverse, the result depends on the assumed alignment rather than on the grounding targets per se.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the SIGReg anti-collapse regularizer retained in the forward JEPA objective."},{"cited_title":"OGBench: Benchmarking offline goal-conditioned RL","cited_arxiv_id":null,"evidence_quote":"Provides OGBench-Cube and OGBench-Scene, the datasets used for probe, planning, and open-loop prediction evaluations."}],"review_version":1}