Pith. sign in

REVIEW 4 major objections 5 minor 48 references

TransDex: Pre-training Visuo-Tactile Policy with Point Cloud Reconstruction for Dexterous Manipulation of Transparent Objects

T0 review · 4 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A point-cloud reconstruction pre-training task plus visuo-tactile fusion lets a dexterous robot hand manipulate transparent objects with an average 83% success rate, the paper reports.

desk verdict TransDex has a promising new architecture for transparent-object manipulation, but the headline success-rate margin isn't statistically supported and the sim-to-real transfer of pre-training is under-validated. read the letter →

arxiv 2603.13869 v2 pith:PXOOMLOR submitted 2026-03-14 cs.RO cs.AI

Fengguan Li , Yifan Ma , Chen Qian , Wentao Rao , Weiwei Shang This is my paper · ORCID
classification cs.ROcs.AI
keywords visuo-tactilefusiondexterousmanipulationtransparentobjectspointcloudreconstructionpre-trainingimitationlearningdiffusionpolicyself-attention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that the main obstacle to dexterous manipulation of transparent objects—depth cameras see through them, and multi-fingered hands occlude the view—can be overcome not by adding a separate perception module, but by pre-training a Transformer encoder to reconstruct object shape from noisy, partially masked hand-object point clouds, then using that encoder inside an end-to-end visuo-tactile policy. If true, this would remove the need for depth completion or segmentation steps that are known to fail under occlusion and cause error accumulation. The paper reports real-robot success rates of 100% (pouring), 80% (flask shaking), and 70% (cup rotating), averaging 83%, compared with 47% for the best baseline, and shows that removing pre-training drops the average to 23%. A careful reader would care because this is a concrete path to robust real-world manipulation of transparent objects, an open problem.

What carries the argument

The load-bearing mechanism is the self-supervised point-cloud reconstruction pre-training task. The model takes the hand-object interaction point cloud, adds noise and randomly masks 70% of patches, then predicts, for each query point in a dense 3D grid, whether it lies on the object surface. The positive query points are sampled from the actual object pose in each instance, forcing the encoder to recover both shape and pose. A contrastive loss on the [cls] token sharpens instance discrimination. After pre-training, only the encoder is kept; it gives the downstream policy a perceptual head start that is robust to missing and noisy data. The fusion of features via multi-round self-attention o

What would settle it

A concrete test is to run the TransDex pipeline on the real robot but replace the pre-trained encoder's weights with an encoder initialized randomly and trained on the same downstream data. The paper reports this drops average success from 83% to 23%. The critical question is why. To test the mechanism directly, one could probe the pre-trained encoder on held-out real hand-object point clouds with ground-truth object meshes and compute the Chamfer distance of its reconstruction. If the real-world reconstruction error is not substantially better than a baseline that simply outputs the mean shap

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that (1) a Transformer trained to classify dense query points as inside or outside a transparent object from a masked, noisy hand-object point cloud learns robust 3D shape and pose features; (2) when that encoder is plugged into a policy that separately fuses global point cloud, hand-object point cloud, tactile keypoints, and tactile force images via self-attention and cross-attention, and then predicts arm trajectory and hand actions with two different heads, the resulting TransDex policy achieves an average 83% success rate on three real-world transparent-object tasks; (3) ablations show pre-training contributes 60 percentage points, tactile information 7

Load-bearing premise

The entire advantage of pre-training rests on the assumption that an encoder trained exclusively on simulated point clouds with synthetic noise and masking will, without any real-world adaptation, extract features that transfer to the very different noise and sparsity statistics of real depth cameras and tactile sensors; the paper's own limitation note says such transfer 'still requires further adaptation.'

Editorial extensions

If this is right

  • If the claim holds, transparent-object manipulation no longer requires a separate depth-completion or segmentation module, eliminating a source of error accumulation.
  • Pre-training on simulated object point clouds can provide a substantial (60-point) boost to a real-world policy, suggesting that reconstruction-style self-supervision is an effective inductive bias.
  • Differentiating the arm and hand action heads according to which fused feature guides them (visual-spatial vs. touch-modulated) improves coordination during contact-rich tasks.
  • The methodology should transfer to other visually challenging object classes (specular, reflective, or partially occluded) as long as the pre-training data covers those geometries.
  • The reported 100% success in pouring and 80% with unseen objects indicates that the policy learns grasp-continuous manipulation rather than a brittle pre-programmed sequence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 60-point contribution of pre-training is so large that it invites a direct test: if the pre-trained encoder's reconstruction accuracy on real point clouds is poor, the success-rate gain may come from the encoder's regularization or feature extraction rather than from genuine 3D understanding; the paper's own evidence for real-world reconstruction is only qualitative.
  • Because the pre-training data is simulated, a natural extension is to replace it with a real hand-object interaction dataset or to add domain randomization; the paper acknowledges 'further adaptation' is needed, which could be a limiting factor in deployment to very different hand geometries or camera placements.
  • The shaking task counts '0.5 success' for small-amplitude shaking if the solution discolors; this soft criterion may inflate reported success rates relative to tasks with strict completion conditions, so direct comparisons across papers should check the success definition.
  • The small number of trials per condition (five per scenario in generalization tests) leaves uncertainty about the precision of the success-rate differences; a larger trial count would make the ablation gaps more convincing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes TransDex, a visuo-tactile policy for dexterous manipulation of transparent objects. The method first pre-trains a Transformer encoder on a point cloud reconstruction task using simulated, noise-corrupted, and randomly masked hand-object interaction point clouds. The pre-trained encoder is then used in a downstream imitation learning policy that combines global point clouds, tactile points, and array tactile force images through hierarchical encoding, multi-round self/cross-attention fusion, and differentiated action heads for the arm and dexterous hand. Real-robot experiments on pouring, shaking, and rotating transparent objects report an average success rate of 83%, outperforming three point-cloud baselines, and ablations show large drops when removing pre-training, tactile information, or differentiated prediction. Additional generalization experiments consider complex backgrounds, lighting changes, and unseen objects.

Significance. If the reported results are reliable, TransDex is a meaningful contribution: it is an end-to-end visuo-tactile policy that avoids explicit perception-completion modules, introduces point cloud reconstruction as self-supervised pre-training for transparent-object manipulation, and provides evidence for differentiated arm/hand action heads. The paper includes real-robot experiments, ablations, and generalization tests. However, the central empirical claim is currently supported only by success-rate point estimates without trial counts, confidence intervals, or significance tests, and the quantitative validation of the pre-training contribution is limited to in-distribution simulation. These gaps need to be addressed before the strength of the claims can be accepted.

major comments (4)
  1. [§IV-D, Table I] The central claim that TransDex is 'significantly higher' than all baselines is not supported by the reported evidence. No trial counts, confidence intervals, or significance tests are given for any entry in Tables I–III. Section IV-G mentions 'five attempts' per generalization scenario, suggesting the main evaluations may have similarly small N. With N=10 per condition, the 83%-vs-47% margin yields p≈0.10 in a two-proportion test; with N=5, p≈0.24. The ablation margin in Table II (83% vs 23%) is equally sensitive to one or two trials. Please report per-condition counts and provide exact tests/confidence intervals for all headline comparisons.
  2. [§IV-E, Table II, §V] The contribution of the pre-training is measured quantitatively only in simulation. The CD-l2 of 8.1e-3 is computed in-distribution on the PyBullet/ClearPose setup; real-world reconstruction is shown only qualitatively in Fig. 5. Yet Table II attributes a 60-percentage-point success-rate drop to removing pre-training. This presumes effective zero-shot sim-to-real transfer of the encoder. The paper's own limitation statement (Section V) says pre-training 'still requires further adaptation and adjustment' for more complex environments. Please provide quantitative real-world reconstruction metrics or an additional controlled experiment isolating the pre-trained encoder's effect on real point clouds.
  3. [§IV-B] The shaking task counts a '0.5 success' for small-amplitude shaking that still causes discolouration. This partial-credit convention is not accompanied by an explanation of how the 50% values are aggregated into the reported success rates, nor whether scoring was performed blindly or by multiple raters. Since the averaging across tasks is central to the headline 83% and the ablations, this scoring subjectivity should be addressed.
  4. [§IV-C] The baseline comparisons modify 3D-ViTac and 3DTacDex (e.g., 3D force for 3D-ViTac, point cloud input for 3DTacDex-P) without giving the exact adaptation procedure or hyperparameter tuning details. The fairness of the comparison depends on these modifications being faithful. Please specify the implementation and tuning budget for each baseline.
minor comments (5)
  1. [§III-B4] 'siutable' should be 'suitable'.
  2. [§IV-E] The phrase '60% higher' is ambiguous; it should be '60 percentage points higher' (23% to 83%).
  3. [§IV-G] Please report the number of trials for each cell in Tables I–III, not just the generalization settings.
  4. [§IV-E] The reconstruction threshold λ in the definition of P_rec is not specified; please give the value used in the visualization and any sensitivity analysis.
  5. [§III-A] The pre-training dataset synthesis is described qualitatively (object size, noise direction, percentage, magnitude varied). Providing the exact ranges and the number of generated scenes would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: central results are external real-robot benchmarks and the pre-training contribution is measured by ablation, not by construction.

full rationale

TransDex's central claims are benchmarked externally: Table I compares success rates against 3D-ViTac, DP3, and 3DTacDex-P on a real robot, so the headline result is not internally forced by the model's equations. The pre-training contribution (Table II, 83% vs 23%) is an ablation of the same policy with and without the pre-trained encoder, not a prediction derived from the pre-training objective; the pre-training loss (Eq. 3) is a reconstruction/contrastive objective defined on simulated ClearPose grasp scenes, and downstream manipulation success is measured separately in the real world. The real-world reconstruction visualizations (Fig. 5) and the paper's own limitation that the pre-training phase relies on simulated data and 'still requires further adaptation and adjustment' concern sim-to-real transfer validity, not circularity. The only overlapping self-citation, [47], describes the dexterous hand hardware; it is not used to justify the policy's perceptual, predictive, or comparative claims. No load-bearing step reduces to its own input by construction. The phrase 'significantly higher' lacks reported trial counts, confidence intervals, or significance tests, but that is a statistical-evidence concern, not evidence of circular reasoning.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced; the method uses existing tactile sensors, a standard point-cloud encoder-decoder, and diffusion heads. The free parameters are hyperparameters whose values are not reported, so their effect on the results cannot be audited. The axioms are domain assumptions about sim-to-real transfer and the effectiveness of the proposed architectural choices.

free parameters (4)
  • λ (contrast-loss weighting in Eq. 3)
    Weight balancing reconstruction classification against contrastive loss; value and sensitivity analysis are not provided.
  • Mask ratio R_mask = 70% used in reconstruction evaluation; training value unspecified
    Random masking ratio for pre-training; chosen by hand, no sweep reported for its downstream effect.
  • λ1 and λ2 (loss weights in Eq. 11)
    Weights for arm and hand loss terms; values are not reported.
  • Reconstruction threshold λ for P_rec
    Confidence threshold used to filter query points in §IV-E; value not reported.
assumptions (4)
  • domain assumption Simulated hand-object interaction point clouds (PyBullet/ClearPose) are representative of real sensor point clouds of transparent objects under noise and occlusion.
    Pre-training data is synthetic (§III-A), and §IV-E claims zero-shot transfer to real scenes with only qualitative support; the conclusion acknowledges further adaptation is needed.
  • domain assumption Masked point-cloud reconstruction (occupancy query classification) is a sufficient pretext task for learning shape/pose features useful for downstream visuo-tactile policy learning.
    This is the core design choice in §III-A; it is empirically supported only by the downstream ablation in §IV-F, not by direct evidence that the learned features encode object shape and pose.
  • domain assumption Cross-attention with tactile-force features as query effectively modulates visual features for dexterous-hand control.
    Stated as a design rationale in §III-B.3; the ablation only shows that the overall policy degrades without tactile information, not that this specific attention mechanism is responsible.
  • standard math Standard DDPM denoising (Eq. 8) reproduces the demonstration action distribution when conditioned on fused features.
    Imported from diffusion policy literature [12,46]; assumed valid here without additional formal justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TransDex: Pre-training Visuo-Tactile Policy with Point Cloud Reconstruction for Dexterous Manipulation of Transparent Objects." pith.science (2026). https://pith.science/paper/PXOOMLOR

@misc{pith2026260313869,
  author       = {Pith},
  title        = {Pith review of: TransDex: Pre-training Visuo-Tactile Policy with Point Cloud Reconstruction for Dexterous Manipulation of Transparent Objects},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PXOOMLOR}},
  note         = {Machine review of arXiv:2603.13869}
}
read the original abstract

Dexterous manipulation enables complex tasks but suffers from self-occlusion, severe depth noise, and depth information loss when manipulating transparent objects. To solve this problem, this paper proposes TransDex, a 3D visuo-tactile fusion motor policy based on point cloud reconstruction pre-training. Specifically, we first propose a self-supervised point cloud reconstruction pre-training approach based on Transformer. This method accurately recovers the 3D structure of objects from interactive point clouds of dexterous hands, even when random noise and large-scale masking are added. Building on this, TransDex is constructed in which perceptual encoding adopts a fine-grained hierarchical scheme and multi-round attention mechanisms adaptively fuse features of the robotic arm and dexterous hand to enable differentiated motion prediction. Results from transparent object manipulation experiments conducted on a real robotic system demonstrate that TransDex outperforms existing baseline methods. Further analysis validates the generalization capabilities of TransDex and the effectiveness of its individual components.

Figures

Figures reproduced from arXiv: 2603.13869 by the authors.

Figure 1
Figure 1. The Overall Framework of the Pre-Training-Based Visual-Tactile Fusion Motor Policy, TransDex. Perceptual information enters the encoder through a fine-grained, hierarchical manner. The hand-object interaction point cloud utilizes a pre-trained encoder, followed by an attention fusion module and two policy heads to achieve feature integration and differentiated action prediction. manipulation due to the open-loop gra… view at source ↗
Figure 2
Figure 2. Pre-Training Framework. First, noise is added to and masked from the original hand-object interaction point cloud to generate input data. Then, features are extracted using a pre-encoder and a Transformer encoder. Finally, the generated query points and a Transformer decoder are employed to accomplish the point cloud reconstruction task for the object. of vision and tactile sensing. A common fusion approach involves… view at source ↗
Figure 3
Figure 3. Robotic System Setup: 1 a 16-DOF dexterous hand, with array tactile sensors equipped on the fingertips and finger pads; 2 a 7-DOF robotic arm; 3 depth cameras; 4 experimental items; 5 a data glove; 6 a motion capture camera. ah k−1 =αk  ah k − γkϵθ  ah k , k, F(M) c  + σkN (0, I) (8) where αk, γk and σk are parameters determined by the noise scheduler and the denoising step k, ϵθ represents the denoising network… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visualization of Policy’s rollout on Three Transparent Object Manipulation Tasks, including pouring, shaking, and rotating. The unseen objects used in the test, as well as the complex backgrounds and lighting conditions, are shown at the bottom. TABLE I SUCCESS RATE OF…
Figure 5
Figure 5. Figure 5: Visualization of Point Cloud Reconstruction Performance in Pre￾Training Tasks and Real-World Transfer Reconstruction Outcomes. robust object representations from sparse point clouds. It exhibits strong robustness against incomplete point cloud information and demonstra…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 4 linked inside Pith

  1. [1]

    See to touch: Learning tactile dexterity through visual incentives,

    I. Guzey, Y . Dai, B. Evans, S. Chintala, and L. Pinto, “See to touch: Learning tactile dexterity through visual incentives,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 13 825–13 832

    2024

  2. [2]

    Dexforce: Extracting force-informed actions from kinesthetic demonstrations for dexterous manipulation,

    C. Chen, Z. Yu, H. Choi, M. Cutkosky, and J. Bohg, “Dexforce: Extracting force-informed actions from kinesthetic demonstrations for dexterous manipulation,”IEEE Robotics and Automation Letters, 2025

    2025

  3. [3]

    Eyesight hand: Design of a fully-actuated dexterous robot hand with integrated vision- based tactile sensors and compliant actuation,

    B. Romero, H.-S. Fang, P. Agrawal, and E. Adelson, “Eyesight hand: Design of a fully-actuated dexterous robot hand with integrated vision- based tactile sensors and compliant actuation,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 1853–1860. 10

    2024

  4. [4]

    Masked visual- tactile pre-training for robot manipulation,

    Q. Liu, Q. Ye, Z. Sun, Y . Cui, G. Li, and J. Chen, “Masked visual- tactile pre-training for robot manipulation,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 13 859–13 875

    2024

  5. [5]

    Multimodal tactile sensing fused with vision for dexterous robotic housekeeping,

    Q. Mao, Z. Liao, J. Yuan, and R. Zhu, “Multimodal tactile sensing fused with vision for dexterous robotic housekeeping,”Nature Commu- nications, vol. 15, no. 1, p. 6871, 2024

    2024

  6. [6]

    Dexterity from touch: Self-supervised pre-training of tactile representations with robotic play,

    I. Guzey, B. Evans, S. Chintala, and L. Pinto, “Dexterity from touch: Self-supervised pre-training of tactile representations with robotic play,” inConference on Robot Learning. PMLR, 2023, pp. 3142–3166

    2023

  7. [7]

    Equibot: Sim (3)-equivariant diffusion policy for generalizable and data efficient learning,

    J. Yang, Z. Cao, C. Deng, R. Antonova, S. Song, and J. Bohg, “Equibot: Sim (3)-equivariant diffusion policy for generalizable and data efficient learning,” inConference on Robot Learning. PMLR, 2025, pp. 1048– 1068

    2025

  8. [8]

    Any-point trajectory modeling for policy learning,

    C. Wen, X. Lin, J. So, K. Chen, Q. Dou, Y . Gao, and P. Abbeel, “Any-point trajectory modeling for policy learning,”arXiv preprint arXiv:2401.00025, 2023

    arXiv 2023

Show all 48 references
  1. [9]

    Integrating with multimodal information for enhancing robotic grasp- ing with vision-language models,

    Z. Zhao, D. Zheng, Y . Chen, J. Luo, Y . Wang, P. Huang, and C. Yang, “Integrating with multimodal information for enhancing robotic grasp- ing with vision-language models,”IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 13 073–13 086, 2025

    2025

  2. [10]

    Maniptrans: Efficient dexterous bimanual manipulation transfer via residual learning,

    K. Li, P. Li, T. Liu, Y . Li, and S. Huang, “Maniptrans: Efficient dexterous bimanual manipulation transfer via residual learning,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 6991–7003

    2025

  3. [11]

    Chemistry3d: Robotic interaction toolkit for chemistry experiments,

    S. Li, Y . Huang, C. Guo, T. Wu, J. Zhang, L. Zhang, and W. Ding, “Chemistry3d: Robotic interaction toolkit for chemistry experiments,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 8064–8071

    2025

  4. [12]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,”The International Journal of Robotics Research, vol. 44, no. 10-11, pp. 1684–1704, 2025

    2025

  5. [13]

    Universal manipulation interface: In-the-wild robot teach- ing without in-the-wild robots,

    C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song, “Universal manipulation interface: In-the-wild robot teach- ing without in-the-wild robots,”arXiv preprint arXiv:2402.10329, 2024

    2024 arXiv

  6. [14]

    3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations,

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu, “3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations,” inProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  7. [15]

    3d-vitac: Learning fine-grained manipulation with visuo-tactile sensing,

    B. Huang, Y . Wang, X. Yang, Y . Luo, and Y . Li, “3d-vitac: Learning fine-grained manipulation with visuo-tactile sensing,” in8th Annual Conference on Robot Learning, 2024

    2024

  8. [16]

    Cordvip: Correspondence-based visuomo- tor policy for dexterous manipulation in real-world,

    Y . Fu, Q. Feng, N. Chen, Z. Zhou, M. Liu, M. Wu, T. Chen, S. Rong, J. Liu, H. Donget al., “Cordvip: Correspondence-based visuomo- tor policy for dexterous manipulation in real-world,”arXiv preprint arXiv:2502.08449, 2025

    2025 arXiv

  9. [17]

    Adaptive visuo-tactile fusion with predictive force attention for dexterous manipulation,

    J. Li, T. Wu, J. Zhang, Z. Chen, H. Jin, M. Wu, Y . Shen, Y . Yang, and H. Dong, “Adaptive visuo-tactile fusion with predictive force attention for dexterous manipulation,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025, pp. 3232–3239

    2025

  10. [18]

    Robot synesthesia: In-hand manipulation with visuotactile sensing,

    Y . Yuan, H. Che, Y . Qin, B. Huang, Z.-H. Yin, K.-W. Lee, Y . Wu, S.-C. Lim, and X. Wang, “Robot synesthesia: In-hand manipulation with visuotactile sensing,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 6558–6565

    2024

  11. [19]

    Generalizable humanoid manipulation with 3d diffusion policies,

    Y . Ze, Z. Chen, W. Wang, T. Chen, X. He, Y . Yuan, X. B. Peng, and J. Wu, “Generalizable humanoid manipulation with 3d diffusion policies,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025, pp. 2873–2880

    2025

  12. [20]

    Realdex: Towards human- like grasping for robotic dexterous hand,

    Y . Liu, Y . Yang, Y . Wang, X. Wu, J. Wang, Y . Yao, S. Schwertfeger, S. Yang, W. Wang, J. Yu, X. He, and Y . Ma, “Realdex: Towards human- like grasping for robotic dexterous hand,” inProceedings of the Thirty- Third International Joint Conference on Artificial Intelligence, ...

    2024

  13. [21]

    Robot dexterous grasping in cluttered scenes based on single-view point cloud,

    Q. Zhao, M. Zheng, Z. Li, S. Huang, and W. Shi, “Robot dexterous grasping in cluttered scenes based on single-view point cloud,”IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 17 160–17 169, 2025

    2025

  14. [22]

    Digger finger: Gelsight tactile sensor for object identification inside granular media,

    R. Patel, R. Ouyang, B. Romero, and E. Adelson, “Digger finger: Gelsight tactile sensor for object identification inside granular media,” inInternational Symposium on Experimental Robotics. Springer, 2020, pp. 105–115

    2020

  15. [23]

    Covering a robot fingertip with uskin: A soft electronic skin with distributed 3-axis force sensitive elements for robot hands,

    T. P. Tomo, A. Schmitz, W. K. Wong, H. Kristanto, S. Somlor, J. Hwang, L. Jamone, and S. Sugano, “Covering a robot fingertip with uskin: A soft electronic skin with distributed 3-axis force sensitive elements for robot hands,”IEEE Robotics and Automation Letters, vol. 3, no. 1...

    2017

  16. [24]

    Paxini: High-precision multi-dimensional tactile sensors & humanoid robots

    Paxini Intelligent Robot, “Paxini: High-precision multi-dimensional tactile sensors & humanoid robots.” [Online]. Available: https: //www.paxini.com/

  17. [25]

    Canonical representation and force-based pretraining of 3d tactile for dexterous visuo-tactile policy learning,

    T. Wu, J. Li, J. Zhang, M. Wu, and H. Dong, “Canonical representation and force-based pretraining of 3d tactile for dexterous visuo-tactile policy learning,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 6786–6792

    2025

  18. [26]

    Robotic perception of transparent objects: A review,

    J. Jiang, G. Cao, J. Deng, T.-T. Do, and S. Luo, “Robotic perception of transparent objects: A review,”IEEE Transactions on Artificial Intelligence, vol. 5, no. 6, pp. 2547–2567, 2023

    2023

  19. [27]

    Clear grasp: 3d shape estimation of transparent objects for manipulation,

    S. Sajjan, M. Moore, M. Pan, G. Nagaraja, J. Lee, A. Zeng, and S. Song, “Clear grasp: 3d shape estimation of transparent objects for manipulation,” in2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 3634–3642

    2020

  20. [28]

    Domain randomization-enhanced depth simulation and restoration for perceiving and grasping specular and transparent objects,

    Q. Dai, J. Zhang, Q. Li, T. Wu, H. Dong, Z. Liu, P. Tan, and H. Wang, “Domain randomization-enhanced depth simulation and restoration for perceiving and grasping specular and transparent objects,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 374–391

    2022

  21. [29]

    A4t: Hierarchical affordance detection for transparent objects depth reconstruction and manipulation,

    J. Jiang, G. Cao, T.-T. Do, and S. Luo, “A4t: Hierarchical affordance detection for transparent objects depth reconstruction and manipulation,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 9826–9833, 2022

    2022

  22. [30]

    Visual–tactile fusion for transparent object grasping in complex backgrounds,

    S. Li, H. Yu, W. Ding, H. Liu, L. Ye, C. Xia, X. Wang, and X.-P. Zhang, “Visual–tactile fusion for transparent object grasping in complex backgrounds,”IEEE Transactions on Robotics, vol. 39, no. 5, pp. 3838– 3856, 2023

    2023

  23. [31]

    Cleardepth: enhanced stereo perception of transparent objects for robotic manipulation,

    K. Bai, H. Zeng, L. Zhang, Y . Liu, H. Xu, Z. Chen, and J. Zhang, “Cleardepth: enhanced stereo perception of transparent objects for robotic manipulation,”arXiv preprint arXiv:2409.08926, 2024

    2024

  24. [32]

    Tcrnet: Transparent object depth completion with cascade refinements,

    D.-H. Zhai, S. Yu, W. Wang, Y . Guan, and Y . Xia, “Tcrnet: Transparent object depth completion with cascade refinements,”IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 1893–1912, 2025

    1912

  25. [33]

    Transparent object depth perception network for robotic manipulation based on orientation-aware guidance and texture enhancement,

    Y . Yan, H. Tian, K. Song, Y . Li, Y . Man, and L. Tong, “Transparent object depth perception network for robotic manipulation based on orientation-aware guidance and texture enhancement,”IEEE Transac- tions on Instrumentation and Measurement, vol. 73, pp. 1–11, 2024

    2024

  26. [34]

    Vtao-bimanip: Masked visual-tactile-action pre-training with object understanding for bimanual dexterous manipulation,

    Z. Sun, Z. Shi, J. Chen, Q. Liu, Y . Cui, J. Chen, and Q. Ye, “Vtao-bimanip: Masked visual-tactile-action pre-training with object understanding for bimanual dexterous manipulation,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2025

    2025

  27. [35]

    Visual-tactile sensing for in-hand object reconstruction,

    W. Xu, Z. Yu, H. Xue, R. Ye, S. Yao, and C. Lu, “Visual-tactile sensing for in-hand object reconstruction,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 8803–8812

    2023

  28. [36]

    Capturing forceful interaction with deformable objects using a deep learning-powered stretchable tactile array,

    C. Jiang, W. Xu, Y . Li, Z. Yu, L. Wang, X. Hu, Z. Xie, Q. Liu, B. Yang, X. Wanget al., “Capturing forceful interaction with deformable objects using a deep learning-powered stretchable tactile array,”Nature Communications, vol. 15, no. 1, p. 9513, 2024

    2024

  29. [37]

    Touch in the wild: Learning fine-grained manipulation with a portable visuo-tactile gripper,

    X. Zhu, B. Huang, and Y . Li, “Touch in the wild: Learning fine-grained manipulation with a portable visuo-tactile gripper,”arXiv preprint arXiv:2507.15062, 2025

    2025

  30. [38]

    Scaling proprioceptive- visual learning with heterogeneous pre-trained transformers,

    L. Wang, X. Chen, J. Zhao, and K. He, “Scaling proprioceptive- visual learning with heterogeneous pre-trained transformers,”Advances in Neural Information Processing Systems, vol. 37, pp. 124 420–124 450, 2024

    2024

  31. [39]

    Low fidelity visuo-tactile pre- training improves vision-only manipulation performance,

    S. Gano, A. George, and A. B. Farimani, “Low fidelity visuo-tactile pre- training improves vision-only manipulation performance,”arXiv preprint arXiv:2406.15639, 2024

    2024 arXiv

  32. [40]

    R3m: A universal visual representation for robot manipulation,

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta, “R3m: A universal visual representation for robot manipulation,” inConference on Robot Learning. PMLR, 2023, pp. 892–909

    2023

  33. [41]

    Dnact: Diffusion guided multi-task 3d policy learning,

    G. Yan, Y .-H. Wu, and X. Wang, “Dnact: Diffusion guided multi-task 3d policy learning,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025, pp. 9464–9471

    2025

  34. [42]

    H-index: Visual reinforcement learning with hand-informed representations for dexterous manipulation,

    Y . Ze, Y . Liu, R. Shi, J. Qin, Z. Yuan, J. Wang, and H. Xu, “H-index: Visual reinforcement learning with hand-informed representations for dexterous manipulation,”Advances in Neural Information Processing Systems, vol. 36, pp. 74 394–74 409, 2023

    2023

  35. [43]

    Learning to manipulate anywhere: A visual generalizable framework for reinforce- ment learning,

    Z. Yuan, T. Wei, S. Cheng, G. Zhang, Y . Chen, and H. Xu, “Learning to manipulate anywhere: A visual generalizable framework for reinforce- ment learning,” inConference on Robot Learning. PMLR, 2025, pp. 1815–1833

    2025

  36. [44]

    Pybullet, a python module for physics simulation for games, robotics and machine learning,

    E. Coumans and Y . Bai, “Pybullet, a python module for physics simulation for games, robotics and machine learning,” 2016

    2016

  37. [45]

    Clearpose: Large-scale transparent object dataset and benchmark,

    X. Chen, H. Zhang, Z. Yu, A. Opipari, and O. Chadwicke Jenkins, 11 “Clearpose: Large-scale transparent object dataset and benchmark,” in European Conference on Computer Vision. Springer, 2022, pp. 381– 396

    2022

  38. [46]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in Neural Information Processing Systems, vol. 33, pp. 6840– 6851, 2020

    2020

  39. [47]

    Development of a flexible cable-driven dexterous hand and arm–hand integrated system,

    Y . Ma, W. Shang, F. Zhang, S. Pang, S. Dai, and S. Cong, “Development of a flexible cable-driven dexterous hand and arm–hand integrated system,”IEEE/ASME Transactions on Mechatronics, 2025

    2025

  40. [48]

    Rise: 3d perception makes real-world robot imitation simple and effective,

    C. Wang, H. Fang, H.-S. Fang, and C. Lu, “Rise: 3d perception makes real-world robot imitation simple and effective,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 2870–2877

    2024

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.