Pith. sign in

REVIEW 5 major objections 7 minor 4 cited by

Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulation

T0 review · 5 major / 7 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that adding an information-bottleneck penalty to behavior cloning reduces latent redundancy and improves success rates in every tested setting.

desk verdict Broad empirical sweep suggesting IB helps behavior cloning, but per-task beta tuning and oracle checkpoint selection weaken the 'across the board' claim. read the letter →

arxiv 2502.02853 v5 pith:5RCVWDSA submitted 2025-02-05 cs.RO cs.LG

Shuanghao Bai , Wanqi Zhou , Pengxiang Ding , Wei Zhao , Donglin Wang , Badong Chen This is my paper · ORCID
classification cs.ROcs.LG
keywords behaviorcloninginformationbottleneckmutuallatentrepresentationredundancyrobotmanipulationMINEgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that behavior cloning in robot manipulation learns latent representations full of redundant information, and that explicitly penalizing that redundancy improves performance and generalization. It does so by adding an information-bottleneck term $\beta I(X;Z)$ to the standard BC loss, where $X$ is the concatenated multimodal input features and $Z$ is the latent representation, with $I(X;Z)$ estimated by a neural mutual-information estimator. Across CortexBench, LIBERO, and real-world pick-and-place tasks, the paper reports that every vanilla BC method tested improves when this penalty is added, with larger gains in more complex and language-conditioned settings. If correct, current behavior-cloning pipelines could be improved without new data or architectural changes, simply by regularizing latent redundancy.

What carries the argument

The carrying object is the Information Bottleneck Lagrangian $L = \beta I(X;Z) - I(Z;A)$, inserted as a penalty into the BC loss after all modality encoders have been concatenated into $X$. $I(X;Z)$ is estimated by MINE (Mutual Information Neural Estimation), a neural-network lower bound on mutual information built on the Donsker-Varadhan representation of KL divergence. Because raw observations $O$ are multimodal, the bottleneck is applied to the fused feature level $X$, giving the Markov chain $O \to X \to Z \to A$; Theorem 4.3 is the bridge that bounds the difference between optimizing $I(X;Z)$ and optimizing $I(O;Z)$ by a small constant $\delta$, assuming the mutual-information gap in Eq. (12) is bounded. The generalization bounds of Theorems 4.1 and 4.2 are what connect lower mutual information to lower generalization error.

What would settle it

Run BC+IB and a matched-strength control regularizer (for example an L2 penalty on the latent $Z$) on the same tasks; if the two succeed at the same rate, then the mutual-information penalty is not what drives the gains. Alternatively, on a synthetic manipulation task where true redundancy in the raw observations can be computed exactly, check whether BC+IB lowers true $I(O;Z)$ and whether that lowering predicts the improvement; the paper's mechanism predicts both.

Watch

Extended reading notes

Core claim

The central claim is that the latent representation $Z$ produced by a behavior-cloning policy from fused multimodal features $X$ contains substantial redundant information, and compressing it with the IB objective $L_{\mathrm{BC-IB}} = \mathbb{E}[\beta I(x_t,z_t) + \|\pi(x_t)-a_t\|^2]$ improves success rate and generalization. The paper reports that under both spatial and temporal fusion, and with full or partial fine-tuning, every vanilla BC baseline with every visual backbone improves when the penalty is added: for example, ResNet+IB gains 10.01% on DMControl, VC-1+IB gains 4.80% on Meta-World, and BC-VILT+IB gains 7.66% and 9.00% on LIBERO-Goal and LIBERO-Object. It also reports lower estimated $I(X,Z)$, attention focused on task-relevant regions, and gains in few-shot settings. The theoretical component argues that generalization error is bounded by mutual-information terms involving $Z$, so reducing $I(X;Z)$ or $I(X;Z|A)$ should tighten the bound, and Theorem 4.3 claims that applying the bottleneck at the fused-feature level rather than at the raw observation level costs only a bounded gap $\delta$.

Load-bearing premise

The load-bearing premise is that the neural estimate of mutual information between the fused input features and the latent representation is a faithful stand-in for the true redundancy in the original observations, so that shrinking it genuinely removes useless information rather than just adding a generic regularizer.

Editorial extensions

If this is right

  • Existing BC pipelines can be upgraded by adding the MINE-based penalty with no change to backbone, fusion module, or dataset; the paper reports consistent gains in every tested combination.
  • The gains grow with task complexity and input diversity: language-conditioned multi-task benchmarks show larger improvements than single-task benchmarks, and longer histories appear to add redundancy that IB removes.
  • Few-shot training benefits as well: BC-VILT+IB beats vanilla BC-VILT with 1, 5, 10, and 20 demonstrations on LIBERO-Goal.
  • BC+IB lowers estimated $I(X,Z)$ and shifts attention onto the arm and target objects, so the practical effect of the penalty is to suppress task-irrelevant background information.
  • The generalization bounds imply that choosing the most compressed latent that still predicts actions should transfer better; the real-world unseen object-bowl combinations are presented as evidence for this.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same fused-level IB penalty should transfer to larger vision-language-action models and diffusion-policy heads, since those architectures also produce a latent $Z$ after fusion; this is an extension the paper does not test.
  • The size of the IB gain across suites could be used as a diagnostic of dataset redundancy: larger gains on LIBERO-Goal and LIBERO-Object than on LIBERO-Spatial suggest redundancy is higher when tasks differ by object identity and goal rather than by spatial structure.
  • A direct control experiment, replacing the MINE term with a non-information regularizer of matched strength, would separate true compression from generic stabilization; the paper does not run this control.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes BC-IB, an information bottleneck regularizer for behavior cloning that adds a MINE-estimated penalty βI(X;Z) at the fused-feature level to the standard BC loss. The method is evaluated on CortexBench (14 tasks, six visual backbones, two fusion types), LIBERO (four suites), and real-world UR5 pick and place tasks, in both full-shot and few-shot settings. The paper also adapts two existing generalization bounds and states a new theorem (Theorem 4.3) intended to justify compressing the intermediate fused representation X rather than the raw observations O. The central claim is that adding this IB penalty improves success rates across the board and that the improvement is due to reduction of latent redundancy.

Significance. If the central claim holds, BC-IB would be a simple, architecture-agnostic plug-in regularizer that improves BC policies across diverse visual encoders and benchmarks, with implications for how redundancy is handled in robot manipulation. The empirical scope is substantial: per-task CortexBench tables, four LIBERO suites, few-shot experiments, and real-world validation are all valuable assets. The paper also ships practical details such as per-task results and ablation of β. However, the strength of the claim as currently stated is undercut by the evaluation protocol (oracle checkpoint selection and per-task β tuning), the absence of significance testing, and a theoretical argument that is circular at a load-bearing point. These issues are fixable with additional experiments and reanalysis, so the contribution is potentially solid but not yet established.

major comments (5)
  1. [Section 5.1, Tables 1-2, Appendix C.1] The 'across the board' claim of Findings 2 and 6 rests on an evaluation protocol that can manufacture improvements. In Section 5.1 the authors state, for CortexBench, 'we test the model every 5 or 10 epochs and select the model with the highest success rate'; with oscillatory learning curves (Figure 10a) and only three seeds, this oracle selection over epochs can preferentially select unusually favorable checkpoints. Additionally, Appendix C.1 reports per-task tuning of β, meaning the hyperparameter was selected separately for each task, presumably on the same validation trajectories used for reporting. The absence of paired significance tests is critical because many reported gains in Tables 1 and 2 are within one standard deviation of the baseline (e.g., R3M+IB on Adroit: 27.33±3.06 vs 25.33±6.43; ViT+IB on TriFinger: 55.93±2.16 vs 55.57±2.65; BC-Transformer+IB on LIBERO-Spatial: 72.50±10.26 vs 68.00±1.00). A comparison with a prespecified checkpoint rule and a fixed β per method/backbone, together with significance testing, is needed to support the across-the-board claim.
  2. [Section 4.3 and Appendix A, Theorem 4.3] Theorem 4.3 is structurally circular. Equation (12) assumes exactly the mutual information gap the theorem is meant to bound: I(o,z;ϕε_o) - I(o,z;ϕ*_o) ≤ δ/β. Substituting that assumption into Eq. (16) immediately yields Eq. (19), so the theorem does not establish that optimizing I(X;Z) controls I(O;Z); it only restates the gap as an assumption. Since the paper relies on this theorem in Section 4.3 to justify applying the bottleneck at the intermediate feature level rather than at the raw input level, the theoretical support for the central claim is not currently valid. The authors should either provide a proof that derives the gap from structural properties of the encoders or clearly state the theorem as a condition rather than a derived guarantee.
  3. [Figure 6 and Section 5.5] The empirical validation that BC+IB reduces redundancy uses the same MINE estimator that appears in the training loss. The training objective explicitly minimizes the MINE lower bound, so a decrease in the MINE estimate between vanilla BC and BC+IB is partly by construction and does not establish that the true mutual information I(X;Z) is reduced. To support the claim that IB genuinely reduces redundancy, the authors should measure I(X;Z) with an independent estimator (e.g., a different parametric family, a non-parametric binning estimator, or a downstream compression probe) and show that the decrease is not an artifact of the MINE architecture.
  4. [Section 5.2, Findings 2-3] The causal attribution that improvements stem from redundancy reduction rather than generic regularization is not established. The IB penalty is the only change, but β is tuned per task and no comparisons are made against standard regularizers (e.g., weight decay, dropout, feature noise, or a simple L2 penalty on activations) with matched hyperparameter tuning. Given that the reported gains are often small relative to the standard deviations, an ablation against simple regularization is necessary to distinguish the information-theoretic mechanism from generic regularization effects.
  5. [Figure 5 and Section 5.5] The β sensitivity analysis is difficult to reconcile with the claim of stable improvements. Figure 5 shows that only a specific range of β yields improvements, that performance peaks at an 'undetermined value,' and that the optimal β appears to vary across task suites. This is at odds with the statement that 'β around 1e-4 consistently yields stable improvements' and raises concern that the reported gains depend on favorable per-suite β choices. The authors should report β sensitivity for each method family and provide a prespecified selection rule, rather than reporting only the favorable configuration.
minor comments (7)
  1. [Section 4.3, Theorem 4.3] The composition order in the definition φo = f ◦ φ appears reversed: if the Markov chain is o → x → z with x = f(o) and z = φ(x), then z = (φ ∘ f)(o), so φo should be φ ∘ f, not f ∘ φ.
  2. [Equations (5)-(6)] Equation (5) is written as L = βI(X;Z) - I(Z;A), but Equation (6) includes only the βI(X;Z) penalty with the BC loss; the relationship between the two objectives should be clarified, since no estimator for I(Z;A) is used in the final loss.
  3. [Appendix C.1, Tables 6-8] The appendix states 'We provide the task-wise results and corresponding β values for CortexBench,' but Tables 6, 7, and 8 do not include β values; either add them to the tables or remove the claim.
  4. [Appendix C.3.1] The reported β for DP+IB is set to 1e-5, which is outside the range [1e-4, 1e-2] stated in Section 5.1 for the Lagrange multiplier; please reconcile this inconsistency.
  5. [Table 2] The last row of Table 2 uses 'BC+VILT+IB' while all other rows and the text use 'BC-VILT+IB'; the notation should be consistent.
  6. [Section 5.4, Figure 4] The real-world results are presented only as bar charts without numerical values, standard deviations, or the number of evaluation runs per condition; numerical reporting would allow verification of the claims.
  7. [Abstract and Introduction] The manuscript claims 'first comprehensive study' and 'first attempt,' but the Related Work already cites Wen et al. (2020), which applies information bottleneck ideas to imitation learning; the novelty claim should be qualified to specify what is new relative to that prior work.

Circularity Check

2 steps flagged · score 5.0 of 10

Theorem 4.3 assumes its own conclusion and Figure 6 reuses the training-time MINE estimator; success-rate improvements remain external and non-circular.

  1. self definitional [Section 4.3, Theorem 4.3, Eqs. (10)-(13); Appendix A, Eqs. (14)-(19)]
    "Assume the mutual information gap satisfies the following condition: for any δ, we have I(o, z; ϕε o) − I(o, z; ϕ∗ o) ≤ δ β . (12) Then, the gap between the two optimizations is bounded as: |J ⋆ − J ε| ≤δ. (13)"

    The theorem's conclusion (13) is obtained by substituting its own assumption (12) into the asserted bound (16): |J ε − J ⋆| ≤ β·(I(o,z;ϕε_o) − I(o,z;ϕ⋆_o)) ≤ β·(δ/β) = δ. Thus the result is the assumption rescaled, not an independent proof that optimizing I(X;Z) controls I(O;Z). The proof also treats the objective of problem (10), defined with log P_φ(z|x), as if it were I(o,z;φ_o) − J/β in Eq. (15), conflating the I(X;Z)-type objective with I(O;Z). The theoretical justification therefore reduces to its own premise plus an unjustified equivalence.

  2. fitted input called prediction [Section 5.5, Figure 6; Eq. (6) and Section 3.2]
    "L_BC−IB = E(xt,at)∼De [ β I(xt, zt) + ∥π(xt) − at∥2 ] (6) ... As shown in Figure 6, BC+IB achieves a larger reduction in I(X, Z) compared to vanilla BC, leading to improved performance and validating the effectiveness of IB. For example, in LIBERO-Goal, IB reduces I(X, Z) to one-quarter of its original value and yields a 7.7% increase in success rate."

    The quantity plotted in Figure 6 as evidence that redundancy is reduced is the same MINE-estimated I(X;Z) that is directly minimized in the IB training loss (Eq. 6), with Section 3.2 specifying MINE as the estimator. Since the training objective penalizes this exact estimator, the observed decrease in I(X,Z) for BC+IB is partly by construction and does not by itself establish a reduction in the true mutual information. The paper provides no independent estimator, exact computation, or held-out check to confirm that the effect is not an artifact of gaming the MINE lower bound, so this mechanistic validation is circular, although the external success-rate evaluations are not.

full rationale

The central empirical claim—Finding 2 in Section 5.2, that every BC method with IB outperforms its vanilla counterpart—rests on success-rate comparisons on CortexBench and LIBERO. Those evaluations are external, and nothing in the derivation forces the reported improvements, so the paper is not wholly circular. However, two load-bearing supports are circular. First, Theorem 4.3, invoked to justify applying the bottleneck at the fused-feature level X rather than raw observations O, assumes in Eq. (12) the mutual-information gap that it concludes in Eq. (13); the Appendix proof simply substitutes the assumption, and Eq. (15) conflates an I(X;Z)-type objective with I(O;Z). Second, the paper's validation that IB reduces redundancy (Figure 6) uses the same MINE estimate that appears as the training penalty in Eq. (6), making the observed decrease partly by construction and insufficient to establish a reduction in true mutual information. I also note, as a separate validity concern rather than a circularity, that the CortexBench protocol selects the best checkpoint using validation success trajectories and tunes beta per task (Section 5.1 and Appendix C.1), which can inflate reported gains; this is a statistical selection issue, not a derivation-circle issue. No load-bearing self-citation chain was found: Theorem 4.1 and 4.2 are adapted from external sources, and the authors' own prior papers appear only as related work or baselines.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim depends on beta, the IB strength, which is tuned per task and not fully reported. It also depends on the assumption that MINE's lower bound is a faithful surrogate for true mutual information, and that compressing the fused feature X transfers to compression of the raw observation O. Theorem 4.3, offered as justification for the feature-level bottleneck, is essentially an assumption in disguise. No new entities are introduced.

free parameters (2)
  • Lagrange multiplier beta = 1e-4 to 1e-2, task-specific, not fully reported
    Eq. 6 balances compression and prediction; Appendix C.1 says 'properly tuning beta' is needed for improvements.
  • MINE discriminator loss weight = 0.1
    Table 5 lists loss weight 0.1 for the MINE objective; no sensitivity analysis is provided.
assumptions (4)
  • domain assumption The generalization error bound of Theorem 4.1 (Shwartz-Ziv et al., 2019) applies to the BC setting and is tight enough that minimizing I(X;Z) reduces generalization error.
    Invoked in Section 4.3; no discussion of bound tightness or the counteracting effect on I(Z;A).
  • domain assumption MINE provides a reliable estimate of mutual information suitable for gradient-based minimization in this high-dimensional setting.
    Section 3.2; MINE is a lower bound, known to be high-variance; optimizing a lower bound does not guarantee reducing true MI.
  • ad hoc to paper The optimization gap between feature-level and input-level bottlenecks is small (Theorem 4.3).
    The proof assumes I(o,z;phi_eps_o)-I(o,z;phi*_o) <= delta/beta (Eq. 12), which is exactly the gap the theorem needs to establish; no independent argument is given.
  • domain assumption Concatenated features X preserve the essential structure of O so that I(X;Z) compression transfers to O.
    Stated in Section 4.3 without rigorous justification; the encoders are frozen or partially trained but O to X is deterministic and may discard information.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulation." pith.science (2026). https://pith.science/paper/5RCVWDSA

@misc{pith2026250202853,
  author       = {Pith},
  title        = {Pith review of: Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5RCVWDSA}},
  note         = {Machine review of arXiv:2502.02853}
}
read the original abstract

Behavior Cloning (BC) is a widely adopted visual imitation learning method in robot manipulation. Current BC approaches often enhance generalization by leveraging large datasets and incorporating additional visual and textual modalities to capture more diverse information. However, these methods overlook whether the learned representations contain redundant information and lack a solid theoretical foundation to guide the learning process. To address these limitations, we adopt an information-theoretic perspective and introduce mutual information to quantify and mitigate redundancy in latent representations. Building on this, we incorporate the Information Bottleneck (IB) principle into BC, which extends the idea of reducing redundancy by providing a structured framework for compressing irrelevant information while preserving task-relevant features. This work presents the first comprehensive study on redundancy in latent representations across various methods, backbones, and experimental settings, while extending the generalizability of the IB to BC. Extensive experiments and analyses on the CortexBench and LIBERO benchmarks demonstrate significant performance improvements with IB, underscoring the importance of reducing input data redundancy and highlighting its practical value for more practical applications. Project Page: https://baishuanghao.github.io/BC-IB.github.io.

Figures

Figures reproduced from arXiv: 2502.02853 by the authors.

Figure 1
Figure 1. Policy architecture of BC. Current BC methods [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Model architectures used in this study. Based on feature fusion methods, we categorize the BC methods in robot [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. (a) BC loss variation for ResNet in spatial and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Real-world robot experiments conducted on a tabletop setup with two settings. (a) Left: the experimental setup. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Effect of the Lagrange multiplier β in BC￾VILT+IB across three suites of LIBERO. When β=0, the method reduces to vanilla BC-VILT. cases. In the single-task setting, VC1+IB significantly outperforms VC1 in both the pick and put tasks. In the more challenging language-co…
Figure 6
Figure 6. Figure 6: Comparison of vanilla BC and BC+IB on the LIBERO benchmark in terms of success rate (sr) and mutual [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Comparison of the success rates of BC-VILT+IB [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Visualizations for one task from each suite in CortexBench. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Visualizations for one task from each suite in LIBERO. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Comparison of success rate curves between single-task and multi-task training. [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: IB encourages the model to focus more selectively on task-relevant regions, specifically the robotic arm and the [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Comparison of the success rates of BC-VILT+IB [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VolumeDP: Modeling Volumetric Representation for Manipulation Policy Learning

    cs.RO 2026-03 conditional novelty 6.0 of 10

    An RGB-only diffusion policy that builds an explicit volumetric representation, distills it into spatial tokens, and conditions a multi-token decoder reaches 88.8% average success on LIBERO.

  2. Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation

    cs.RO 2025-08 conditional novelty 6.0 of 10

    Low within-subdataset diversity and large between-subdataset differences cause shortcut learning in generalist robot policies, and targeted augmentation can mitigate it.

  3. SPARSE Data, Rich Results: Few-Shot Semi-Supervised Learning via Class-Conditioned Image Translation

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    A GAN framework that translates unlabeled medical images between classes and fuses ensemble, time-averaged pseudo-labels outperforms six prior GAN semi-supervised methods on MedMNIST at 5-50 labels per class.

  4. Towards Dual-Brain Minimal Sufficient Representation for Vision-Language Navigation

    cs.CV 2026-07 reject novelty 4.0 of 10

    A CP-decomposed, instruction-conditioned latent bottleneck (CompactNav) improves VLN-CE success rate by about 2% over prior state of the art on two benchmarks.

Reference graph

Works this paper leans on

72 extracted references · 41 canonical work pages · cited by 4 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Amjad, R. A. and Geiger, B. C. Learning representations for neural network-based classification using the information bottleneck principle. IEEE transactions on pattern analysis and machine intelligence, 42 0 (9): 0 2225--2239, 2019

    work page 2019

  3. [3]

    D., Chernova, S., Veloso, M., and Browning, B

    Argall, B. D., Chernova, S., Veloso, M., and Browning, B. A survey of robot learning from demonstration. Robotics and autonomous systems, 57 0 (5): 0 469--483, 2009

    work page 2009

  4. [4]

    Dynamic bottleneck for robust self-supervised exploration

    Bai, C., Wang, L., Han, L., Garg, A., Hao, J., Liu, P., and Wang, Z. Dynamic bottleneck for robust self-supervised exploration. Advances in Neural Information Processing Systems, 34: 0 17007--17020, 2021

    work page 2021

  5. [5]

    and Sammut, C

    Bain, M. and Sammut, C. A framework for behavioural cloning. In Machine Intelligence 15, pp.\ 103--129, 1995

    work page 1995

  6. [6]

    Image segmentation using information bottleneck method

    Bardera, A., Rigau, J., Boada, I., Feixas, M., and Sbert, M. Image segmentation using information bottleneck method. IEEE Transactions on Image Processing, 18 0 (7): 0 1601--1612, 2009

    work page 2009

  7. [7]

    I., Baratin, A., Rajeshwar, S., Ozair, S., Bengio, Y., Courville, A., and Hjelm, D

    Belghazi, M. I., Baratin, A., Rajeshwar, S., Ozair, S., Bengio, Y., Courville, A., and Hjelm, D. Mutual information neural estimation. In International conference on machine learning, pp.\ 531--540. PMLR, 2018

    2018

  8. [8]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, 2023

    work page 2023

Show all 72 references
  1. [9]

    I., Yu, T., Finn, C., and Hausman, K

    Burns, K., Witzel, Z., Hamid, J. I., Yu, T., Finn, C., and Hausman, K. What makes pre-trained visual representations successful for robust manipulation? arXiv preprint arXiv:2312.12444, 2023

    2023 arXiv

  2. [10]

    Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation

    Cheang, C.-L., Chen, G., Jing, Y., Kong, T., Li, H., Li, Y., Liu, Y., Wu, H., Xu, J., Yang, Y., et al. Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.06158, 2024

    arXiv

  3. [11]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., Tedrake, R., and Song, S. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, pp.\ 02783649241273668, 2023

    2023

  4. [12]

    Cover, T. M. Elements of information theory. John Wiley & Sons, 1999

    1999

  5. [13]

    Openhelix: A short survey, empirical analysis, and open-source dual-system vla model for robotic manipulation

    Cui, C., Ding, P., Song, W., Bai, S., Tong, X., Ge, Z., Suo, R., Zhou, W., Liu, Y., Jia, B., et al. Openhelix: A short survey, empirical analysis, and open-source dual-system vla model for robotic manipulation. arXiv preprint arXiv:2505.03912, 2025

    2025 arXiv

  6. [14]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.\ 248--255. Ieee, 2009

    2009

  7. [15]

    Quar-vla: Vision-language-action model for quadruped robots

    Ding, P., Zhao, H., Zhang, W., Song, W., Zhang, M., Huang, S., Yang, N., and Wang, D. Quar-vla: Vision-language-action model for quadruped robots. In European Conference on Computer Vision, pp.\ 352--367. Springer, 2024

    2024

  8. [16]

    Donsker, M. D. and Varadhan, S. S. Asymptotic evaluation of certain markov process expectations for large time. iv. Communications on pure and applied mathematics, 36 0 (2): 0 183--212, 1983

    1983

  9. [17]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021

    2021

  10. [18]

    Learning robust representations via multi-view information bottleneck

    Federici, M., Dutta, A., Forr \'e , P., Kushman, N., and Akata, Z. Learning robust representations via multi-view information bottleneck. In International Conference on Learning Representations, 2019

    2019

  11. [19]

    Carp: Visuomotor policy learning via coarse-to-fine autoregressive prediction

    Gong, Z., Ding, P., Lyu, S., Huang, S., Sun, M., Zhao, W., Fan, Z., and Wang, D. Carp: Visuomotor policy learning via coarse-to-fine autoregressive prediction. arXiv preprint arXiv:2412.06782, 2024

    2024 arXiv

  12. [20]

    Rvt: Robotic view transformer for 3d object manipulation

    Goyal, A., Xu, J., Guo, Y., Blukis, V., Chao, Y.-W., and Fox, D. Rvt: Robotic view transformer for 3d object manipulation. In Conference on Robot Learning, pp.\ 694--710. PMLR, 2023

    2023

  13. [21]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\...

    2022

  14. [22]

    Bridging the sim-to-real gap from the information bottleneck perspective

    He, H., Wu, P., Bai, C., Lai, H., Wang, L., Pan, L., Hu, X., and Zhang, W. Bridging the sim-to-real gap from the information bottleneck perspective. In 8th Annual Conference on Robot Learning, 2024

    2024

  15. [23]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016

    2016

  16. [24]

    Masked autoencoders are scalable vision learners

    He, K., Chen, X., Xie, S., Li, Y., Doll \'a r, P., and Girshick, R. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 16000--16009, 2022

    2022

  17. [25]

    Video prediction policy: A generalist robot policy with predictive visual representations

    Hu, Y., Guo, Y., Wang, P., Chen, X., Wang, Y.-J., Zhang, J., Sreenath, K., Lu, C., and Chen, J. Video prediction policy: A generalist robot policy with predictive visual representations. arXiv preprint arXiv:2412.14803, 2024

    2024 arXiv

  18. [26]

    Video prediction policy: A generalist robot policy with predictive visual representations

    Hu, Y., Guo, Y., Wang, P., Chen, X., Wang, Y.-J., Zhang, J., Sreenath, K., Lu, C., and Chen, J. Video prediction policy: A generalist robot policy with predictive visual representations. In International conference on machine learning, 2025

    2025

  19. [27]

    Bc-z: Zero-shot task generalization with robotic imitation learning

    Jang, E., Irpan, A., Khansari, M., Kappler, D., Ebert, F., Lynch, C., Levine, S., and Finn, C. Bc-z: Zero-shot task generalization with robotic imitation learning. In Conference on Robot Learning, pp.\ 991--1002. PMLR, 2022

    2022

  20. [28]

    Ib-gan: Disentangled representation learning with information bottleneck generative adversarial networks

    Jeon, I., Lee, W., Pyeon, M., and Kim, G. Ib-gan: Disentangled representation learning with information bottleneck generative adversarial networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp.\ 7926--7934, 2021

    2021

  21. [29]

    Chain-of-thought predictive control

    Jia, Z., Thumuluri, V., Liu, F., Chen, L., Huang, Z., and Su, H. Chain-of-thought predictive control. In Forty-first International Conference on Machine Learning, 2024

    2024

  22. [30]

    S., Kollar, T., Finn, C., Sadigh, D., and Liang, P

    Karamcheti, S., Nair, S., Chen, A. S., Kollar, T., Finn, C., Sadigh, D., and Liang, P. Language-driven representation learning for robotics. In Robotics: Science and Systems, 2023

    2023

  23. [31]

    How does information bottleneck help deep learning? In International Conference on Machine Learning, pp.\ 16049--16096

    Kawaguchi, K., Deng, Z., Ji, X., and Huang, J. How does information bottleneck help deep learning? In International Conference on Machine Learning, pp.\ 16049--16096. PMLR, 2023

    2023

  24. [32]

    J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al

    Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024

    2024 arXiv

  25. [33]

    Curiosity-bottleneck: Exploration by distilling task-specific novelty

    Kim, Y., Nam, W., Kim, H., Kim, J.-H., and Kim, G. Curiosity-bottleneck: Exploration by distilling task-specific novelty. In International conference on machine learning, pp.\ 3379--3388. PMLR, 2019

    2019

  26. [34]

    Reducing information bottleneck for weakly supervised semantic segmentation

    Lee, J., Choi, J., Mok, J., and Yoon, S. Reducing information bottleneck for weakly supervised semantic segmentation. Advances in neural information processing systems, 34: 0 27408--27421, 2021

    2021

  27. [35]

    Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation

    Li, Q., Liang, Y., Wang, Z., Luo, L., Chen, X., Liao, M., Wei, F., Deng, Y., Xu, S., Zhang, Y., et al. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650, 2024 a

    2024 arXiv

  28. [36]

    Vision-language foundation models as effective robot imitators

    Li, X., Liu, M., Zhang, H., Yu, C., Xu, J., Wu, H., Cheang, C., Jing, Y., Zhang, W., Liu, H., et al. Vision-language foundation models as effective robot imitators. In International Conference on Learning Representations, 2024 b

    2024

  29. [37]

    Libero: Benchmarking knowledge transfer for lifelong robot learning

    Liu, B., Zhu, Y., Gao, C., Feng, Y., Liu, Q., Zhu, Y., and Stone, P. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36, 2024

    2024

  30. [38]

    Contrastive imitation learning for language-guided multi-task robotic manipulation

    Ma, T., Zhou, J., Wang, Z., Qiu, R., and Liang, J. Contrastive imitation learning for language-guided multi-task robotic manipulation. In Conference on Robot Learning, 2024

    2024

  31. [39]

    Where are we in the search for an artificial visual cortex for embodied intelligence? Advances in Neural Information Processing Systems, 36: 0 655--677, 2023

    Majumdar, A., Yadav, K., Arnaud, S., Ma, J., Chen, C., Silwal, S., Jain, A., Berges, V.-P., Wu, T., Vakil, J., et al. Where are we in the search for an artificial visual cortex for embodied intelligence? Advances in Neural Information Processing Systems, 36: 0 655--677, 2023

    2023

  32. [40]

    R3m: A universal visual representation for robot manipulation

    Nair, S., Rajeswaran, A., Kumar, V., Finn, C., and Gupta, A. R3m: A universal visual representation for robot manipulation. In Conference on Robot Learning, pp.\ 892--909. PMLR, 2023

    2023

  33. [41]

    and Majumdar, A

    Pacelli, V. and Majumdar, A. Learning task-driven control policies via information bottlenecks. In Robotics: Science and Systems (RSS), 2020

    2020

  34. [42]

    and Zhu, J

    Pearce, T. and Zhu, J. Counter-strike deathmatch with large-scale behavioural cloning. In 2022 IEEE Conference on Games (CoG), pp.\ 104--111. IEEE, 2022

    2022

  35. [43]

    Film: Visual reasoning with a general conditioning layer

    Perez, E., Strub, F., De Vries, H., Dumoulin, V., and Courville, A. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  36. [44]

    Pomerleau, D. A. Efficient training of artificial neural networks for autonomous navigation. Neural computation, 3 0 (1): 0 88--97, 1991

    1991

  37. [45]

    Real-world robot learning with masked visual pre-training

    Radosavovic, I., Xiao, T., James, S., Abbeel, P., Malik, J., and Darrell, T. Real-world robot learning with masked visual pre-training. In Conference on Robot Learning, pp.\ 416--426. PMLR, 2023

    2023

  38. [46]

    Learning complex dexterous manipulation with deep reinforcement learning and demonstrations

    Rajeswaran, A., Kumar, V., Gupta, A., Vezzani, G., Schulman, J., Todorov, E., and Levine, S. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. Robotics: Science and Systems, 2018

    2018

  39. [47]

    E., Wenzel, F., and Lioutikov, R

    Reuss, M., Ya g murlu, \"O . E., Wenzel, F., and Lioutikov, R. Multimodal diffusion transformer: Learning versatile behavior from multimodal goals. In First Workshop on Vision-Language Models for Navigation and Manipulation at ICRA 2024, 2024

    2024

  40. [48]

    R., Wang, K., Shin, W

    Saxena, V., Bronars, M., Arachchige, N. R., Wang, K., Shin, W. C., Nasiriany, S., Mandlekar, A., and Xu, D. What matters in learning from large-scale datasets for robot manipulation. In International Conference on Learning Representations, 2025

    2025

  41. [49]

    REPRESENTATION COMPRESSION AND GENERALIZATION IN DEEP NEURAL NETWORKS , 2019

    Shwartz-Ziv, R., Painsky, A., and Tishby, N. REPRESENTATION COMPRESSION AND GENERALIZATION IN DEEP NEURAL NETWORKS , 2019. URL https://openreview.net/forum?id=SkeL6sCqK7

    2019

  42. [50]

    Accelerating vision-language-action model integrated with action chunking via parallel decoding

    Song, W., Chen, J., Ding, P., Zhao, H., Zhao, W., Zhong, Z., Ge, Z., Ma, J., and Li, H. Accelerating vision-language-action model integrated with action chunking via parallel decoding. arXiv preprint arXiv:2503.02310, 2025

    2025

  43. [51]

    Tassa, Y., Doron, Y., Muldal, A., Erez, T., Li, Y., Casas, D. d. L., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., et al. Deepmind control suite. arXiv preprint arXiv:1801.00690, 2018

    2018 arXiv

  44. [52]

    Predictive inverse dynamics models are scalable learners for robotic manipulation

    Tian, Y., Yang, S., Zeng, J., Wang, P., Lin, D., Dong, H., and Pang, J. Predictive inverse dynamics models are scalable learners for robotic manipulation. In International Conference on Learning Representations, 2025

    2025

  45. [53]

    C., and Bialek, W

    Tishby, N., Pereira, F. C., and Bialek, W. The information bottleneck method. arXiv preprint physics/0004057, 1999

    1999 arXiv

  46. [54]

    Behavioral cloning from observation

    Torabi, F., Warnell, G., and Stone, P. Behavioral cloning from observation. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp.\ 4950--4957, 2018

    2018

  47. [55]

    Attention is all you need

    Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  48. [56]

    Multi-view information-bottleneck representation learning

    Wan, Z., Zhang, C., Zhu, P., and Hu, Q. Multi-view information-bottleneck representation learning. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp.\ 10085--10092, 2021

    2021

  49. [57]

    Scaling proprioceptive-visual learning with heterogeneous pre-trained transformers

    Wang, L., Chen, X., Zhao, J., and He, K. Scaling proprioceptive-visual learning with heterogeneous pre-trained transformers. In Advances in neural information processing systems, 2024

    2024

  50. [58]

    Fighting copycat agents in behavioral cloning from observation histories

    Wen, C., Lin, J., Darrell, T., Jayaraman, D., and Gao, Y. Fighting copycat agents in behavioral cloning from observation histories. Advances in Neural Information Processing Systems, 33: 0 2564--2575, 2020

    2020

  51. [59]

    Any-point trajectory modeling for policy learning

    Wen, C., Lin, X., So, J., Chen, K., Dou, Q., Gao, Y., and Abbeel, P. Any-point trajectory modeling for policy learning. In Robotics: Science and Systems, 2024 a

    2024

  52. [60]

    Diffusion-vla: Scaling robot foundation models via unified diffusion and autoregression

    Wen, J., Zhu, M., Zhu, Y., Tang, Z., Li, J., Zhou, Z., Li, C., Liu, X., Peng, Y., Shen, C., et al. Diffusion-vla: Scaling robot foundation models via unified diffusion and autoregression. arXiv preprint arXiv:2412.03293, 2024 b

    2024 arXiv

  53. [61]

    Dexvla: Vision-language model with plug-in diffusion expert for general robot control

    Wen, J., Zhu, Y., Li, J., Tang, Z., Shen, C., and Feng, F. Dexvla: Vision-language model with plug-in diffusion expert for general robot control. arXiv preprint arXiv:2502.05855, 2025

    2025 arXiv

  54. [62]

    Unleashing large-scale video generative pre-training for visual robot manipulation

    Wu, H., Jing, Y., Cheang, C., Chen, G., Xu, J., Li, X., Liu, M., Li, H., and Kong, T. Unleashing large-scale video generative pre-training for visual robot manipulation. In International Conference on Learning Representations, 2024

    2024

  55. [63]

    Trifinger: An open-source robot for learning dexterity

    Wuthrich, M., Widmaier, F., Grimminger, F., Joshi, S., Agrawal, V., Hammoud, B., Khadiv, M., Bogdanovic, M., Berenz, V., Viereck, J., et al. Trifinger: An open-source robot for learning dexterity. In Conference on Robot Learning, pp.\ 1871--1882. PMLR, 2021

    2021

  56. [64]

    Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning

    Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., and Levine, S. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on robot learning, pp.\ 1094--1100. PMLR, 2020

    2020

  57. [65]

    Robotic control via embodied chain-of-thought reasoning

    Zawalski, M., Chen, W., Pertsch, K., Mees, O., Finn, C., and Levine, S. Robotic control via embodied chain-of-thought reasoning. In 8th Annual Conference on Robot Learning, 2024

    2024

  58. [66]

    3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations

    Ze, Y., Zhang, G., Zhang, K., Hu, C., Wang, M., and Xu, H. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations. In ICRA 2024 Workshop on 3D Visual Representations for Robot Manipulation, 2024

    2024

  59. [67]

    Learning manipulation by predicting interaction

    Zeng, J., Bu, Q., Wang, B., Xia, W., Chen, L., Dong, H., Song, H., Wang, D., Hu, D., Luo, P., et al. Learning manipulation by predicting interaction. In Robotics: Science and Systems, 2024

    2024

  60. [68]

    Gevrm: Goal-expressive video generation model for robust visual manipulation

    Zhang, H., Ding, P., Lyu, S., Peng, Y., and Wang, D. Gevrm: Goal-expressive video generation model for robust visual manipulation. In International Conference on Learning Representations, 2025 a

    2025

  61. [69]

    Hirt: Enhancing robotic control with hierarchical robot transformers

    Zhang, J., Guo, Y., Chen, X., Wang, Y.-J., Hu, Y., Shi, C., and Chen, J. Hirt: Enhancing robotic control with hierarchical robot transformers. In 8th Annual Conference on Robot Learning, 2024

    2024

  62. [70]

    Up-vla: A unified understanding and prediction model for embodied agent

    Zhang, J., Guo, Y., Hu, Y., Chen, X., Zhu, X., and Chen, J. Up-vla: A unified understanding and prediction model for embodied agent. arXiv preprint arXiv:2501.18867, 2025 b

    2025 arXiv

  63. [71]

    Vlas: Vision-language-action model with speech instructions for customized robot manipulation

    Zhao, W., Ding, P., Zhang, M., Gong, Z., Bai, S., Zhao, H., and Wang, D. Vlas: Vision-language-action model with speech instructions for customized robot manipulation. In International Conference on Learning Representations, 2025

    2025

  64. [72]

    Spa: 3d spatial-awareness enables effective embodied representation

    Zhu, H., Yang, H., Wang, Y., Yang, J., Wang, L., and He, T. Spa: 3d spatial-awareness enables effective embodied representation. arXiv preprint arXiv:2410.08208, 2024

    2024 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.