Pith. sign in

REVIEW 5 major objections 8 minor 2 cited by

ControlTac: Force- and Position-Controlled Tactile Data Augmentation with a Single Reference Image

T0 review · 5 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that tactile images generated from a single reference image, controlled by contact force and position, are realistic enough to augment real tactile data and improve downstream robotic tasks.

desk verdict A genuinely novel controllable tactile image generator with real downstream gains, but the force-invariant contact mask assumption needs validation before the 1-10 N claims hold. read the letter →

arxiv 2505.20498 v2 pith:7ODSNVPB submitted 2025-05-26 cs.CV cs.LGcs.RO

Dongyu Luo , Kelin Yu , Amir-Hossein Shahidzadeh , Cornelia Fermüller , Yiannis Aloimonos , Ruohan Gao This is my paper · ORCID
classification cs.CVcs.LGcs.RO
keywords tactilesensingdataaugmentationconditionaldiffusionmodelcontactforcepositionmaskposeestimationrobotmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Collecting tactile images is expensive because every sample requires physical contact between a sensor and an object, and the images vary across sensors and lighting. This paper claims that a single real tactile reference image, together with two physical control signals—the relative 3D contact force and a 2D contact mask encoding contact position—can generate tens of thousands of realistic tactile images that behave like real data in downstream tasks. The proposed two-stage generator first applies the requested force with a latent diffusion model, then shifts contact position by injecting the mask into the frozen force model. The paper reports consistent gains on force estimation, contact pose estimation, and object classification, with pose error cut roughly in half when training on 12,000 generated images instead of 3,000 real ones. If correct, the method removes the main bottleneck in tactile learning: it turns one physical touch into a large, controllable training set.

What carries the argument

The central mechanism is a contact mask used as the position-control signal. This binary mask marks the approximate contact area between object and sensor, and the paper shifts position by translating and rotating the mask instead of estimating a center point. The assumption that the mask's shape stays fixed once a reference image is given is what lets a single mask encode all positions. The second piece is the two-stage split: a latent diffusion generator learned for force, then a frozen copy of that generator with an auxiliary condition-injection branch trained for position. This split is what allows the position prior, which needs less data, to be learned separately from the force prior rather than jointly.

What would settle it

Measure the same object's contact mask at several forces between 1 N and 10 N with the contact position held fixed; if mask area or outline changes by more than a pixel or two as force grows, the single-mask assumption fails and generated images misrepresent contact geometry at part of the claimed range.

Watch

Extended reading notes

Core claim

The central claim is that physically conditioned generation, not free-form synthesis or simulation, is what makes synthetic tactile data usable, and that force and contact position are the two conditions that carry the weight. Starting from one reference tactile image, a relative 3D force vector $\Delta F$, and a binary contact mask, the generator outputs an image whose gel deformation, texture, and contact location match the requested physical state. The force stage is trained on about 20,000 force-annotated images, and the position stage is trained on about 7,000 mask-aligned examples by attaching a structure-injection branch to the frozen force model. The paper reports that this recipe yields consistent downstream gains: pose estimation error drops from 9 px to 4 px when 12,000 generated images replace 3,000 real ones for a cylinder, an unseen object reaches 4 px error from generated data alone, and classification accuracy rises to 0.95–0.99 across classifiers.

Load-bearing premise

The position-control stage assumes that a fixed binary contact mask, taken from the reference image, stays shape-correct no matter what force is applied, so only its position changes.

Editorial extensions

If this is right

  • Contact pose estimation can be trained almost entirely on generated data: 12,000 generated images cut cylinder X error from 9 px to 4 px, and an unseen T-shape reaches 4 px error with no real training images.
  • Real-data requirements for force estimation shrink by more than half: 8,000 real images plus generated data match the performance of 20,000 real images.
  • Classification of six objects from one reference image each rises to 0.93–0.99 with generated augmentation, far above the 0.60–0.79 range of geometric and color augmentation.
  • The augmented-data models deploy in real manipulation: a 3 mm-tolerance insertion task succeeds 85–90% of the time, and pushing-force estimates track a real-data-trained model within about 0.03 N.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The contact-mask assumption is testable and may be the first thing to break: measuring mask area at 1, 3, 6, and 10 N would reveal whether the same mask can serve all forces.
  • The modular two-stage design suggests texture, hardness, or slip conditions could be added as additional control branches, each trainable on a smaller dataset than a joint model.
  • Because only one reference image is needed, an online variant could generate training data on the fly: a robot touches a new object once, then synthesizes thousands of posed variants for a task-specific model.
  • The reported 4–10 N restriction implies low-force generation is the regime most likely to fail, so practitioners who need fine contact detection below 4 N should validate generated images before trusting them.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. This paper proposes ControlTac, a two-stage conditional diffusion framework for generating tactile images from a single reference tactile image, conditioned on a 3D force vector (stage 1) and on a binary contact mask representing contact position (stage 2, via ControlNet). The generators are trained on the FeelAnyForce dataset. The authors evaluate generation quality with MSE/SSIM against two baselines, and use the generated images for three downstream tasks: 3D force estimation, contact pose estimation, and object classification, plus three real-world robot experiments. The main quantitative claims are that generated data reduce force-estimation MAE in low-data regimes (Fig. 5), that 12,000 generated pose images give lower pose error than 3,000 real images (Table 2), and that ControlTac-based augmentation outperforms traditional geometric/color augmentation on classification (Table 3).

Significance. If substantiated, ControlTac would be a useful tool for scaling tactile datasets with modest annotation cost, and the two-stage design (separating force and position control) is a sensible architecture choice. The paper's strengths include extensive downstream experiments, real-world deployments (pushing, pose tracking, 3mm-tolerance insertion), and explicit comparisons against a physics-based simulator and other generative baselines. The force-estimation results in Fig. 5 show a clear benefit from combining real and generated data. However, the evaluation is concentrated on a single sensor/dataset lineage (FeelAnyForce, with which several authors are affiliated), several central claims outrun the evidence (force-range coverage, augmentation vs synthetic replacement, unseen-object generalization), and the paper does not report code or data release. These issues limit the current support for the abstract's broad claim of 'consistent gains' through data augmentation, though they are addressable by additional experiments and careful re-scoping.

major comments (5)
  1. [Sec. 3.1 (Position-Control Mask)] The assumption that 'for a given object, the shape of the contact mask remains consistent, regardless of the applied force' is asserted without supporting measurements. For a gel-based sensor, the contact patch generally grows with normal load, so a mask taken from the initial-contact reference image may misrepresent the contact geometry at 10 N. Because this fixed mask is used both as the ControlNet conditioning signal and as the annotation for all generated pose/force data, a force-dependent mask would introduce a systematic bias into the generated images and into downstream pose and force estimators. The paper should validate this invariance by comparing real masks across the claimed 1–10 N range (e.g., by thresholding contact regions in FeelAnyForce images at multiple forces) or explicitly restrict the position-control claim to the range where the assumption holds. The elevated 1–10 N MAE in Appendix C.1 (0.35 vs 0.17 for 4–10 N) is consistent with this concern rather than resolving it.
  2. [Sec. 4.2 and Appendix C.1] The full two-stage framework's force-estimation evaluation is restricted to 4–10 N ('we restrict evaluation to the 4–10 N range'), but the abstract and Sec. 1 claim force-controlled generation and 'consistent gains' without this caveat. Table 5 shows that the 1–10 N MAE is roughly double the 4–10 N MAE (0.35 vs 0.17 with 15k generated samples), so the 1–10 N claim is not supported by the full framework. The authors should either provide 1–10 N results for the full framework (including the position-control stage) or explicitly scope the force-range claim in the abstract and introduction.
  3. [Secs. 3.2, 4.3, 4.4] The paper's central claim is that ControlTac 'can effectively augment tactile datasets and lead to consistent gains' (Abstract), but the experiments do not consistently test augmentation (real+generated) versus real-only training. In pose estimation (Table 2), the comparisons are generated-only versus real-only, with no real+generated condition; in classification (Table 3), all training sets are synthetic (geometric/color/ControlTac), with no real-data or real+generated baseline. Only the force-estimation experiments in Fig. 5 and Fig. 6 actually co-train real and generated data. The authors should either add real+generated conditions for pose and classification, or revise the wording to clarify that most experiments demonstrate synthetic-data replacement rather than augmentation.
  4. [Sec. 4.3 (unseen T-shape)] The T-shape experiment is central to the claim of generalization to unseen objects, but the paper does not specify how the T-shape reference image and contact mask are obtained or how the position-control generator, trained only on FeelAnyForce objects, is able to generate T-shape images. Without this detail, it is unclear whether the result demonstrates genuine generalization to a new object shape or a form of overfitting to the mask/reference. Please describe the T-shape generation pipeline, including the source of the reference image and mask, and any fine-tuning or adaptation used.
  5. [Sec. 4.4 (Classification)] The classification experiment lacks a test-set description. The text says 'unseen object classification' and uses six objects (five from FeelAnyForce plus T-shape), but it does not state whether the classifier is tested on real tactile images, generated images, or a separate set of objects. Without knowing the test distribution, the accuracy numbers in Table 3 are not interpretable and the comparison between augmentation methods cannot be assessed. The authors should specify the train/test split and the source of test labels.
minor comments (8)
  1. [Sec. 4.3] The statement that performance 'peaks when using 4,000 image per object' conflicts with Table 2, where 12,000 generated images give better results than 3,000 for cylinder (X error 9→4, Y error 8→5) and 4,000 is the best shown for cross; please clarify the sampling and the reported peak.
  2. [Sec. 4.1] The hybrid baseline is trained on 7,000 samples while the separate pipeline and ControlTac use 20,000 force-control samples plus 7,000 position-control samples; because the baselines are given different amounts of training data, the MSE/SSIM comparison in Table 1 is not apples-to-apples and the textual explanation should be supported by a same-data ablation.
  3. [Secs. 1 and 4.1] Please fix typographical errors, including 'poition' and 'conistent' in Sec. 1, and 'additinoal' and 'posision' in Sec. 4.1.
  4. [Sec. 4.4] The classification section refers to 'Appendix 11'; this should be 'Fig. 11' or 'Appendix F.4'.
  5. [Table 3] The column headers like '2400 (G)' are not defined; please clarify whether the numbers refer to training sample counts and what the augmentation methods mean for each column.
  6. [Sec. 4.5] The real-time pose tracking experiment reports only the update rate (10 Hz) and no quantitative tracking error; adding a metric would strengthen the validation.
  7. [Sec. 3.1] The claimed one-pixel/one-degree alignment precision of the contact masks is not validated; a brief assessment of alignment consistency would be helpful given that masks serve as ground truth for position control.
  8. [Sec. 4.3] The 'fixed' force is described as the median value 6.5 N, but the specific force values used for the 'unfixed' (varying) condition are not reported; please state the force list for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ControlTac is trained on real FeelAnyForce images and its gains are measured on held-out real test sets and real-world tasks; the force-invariant mask is an unvalidated physical prior, not a circular reduction.

full rationale

Walking the claimed derivation chain: the force-control generator y=D(F_f(z(x),ΔF)) is trained on 20,000 real FeelAnyForce images and evaluated by MSE/SSIM on held-out real images (Table 1); the position-control generator y'=D(F_c(z(x),z(c),ΔF)) is fine-tuned on real aligned masks. Downstream force and pose estimators are trained on generated images whose labels are the conditioning force/position inputs, but the paper's reported claims are against real test data—held-out FeelAnyForce positions/forces and newly collected real T-shape poses—and against real ATI force readings in the pushing experiments (Table 4). The 'twice as good' pose statement is therefore an empirical comparison on real data, not an identity forced by construction. Reuse of FeelAnyForce [33], which shares three authors, is dataset/framework reuse rather than load-bearing self-citation: no uniqueness theorem or fitted conclusion is imported to forbid alternatives, and the generator is tested against baselines and real-world pushing/insertion tasks. The Sec. 3.1 assumption that contact-mask shape is invariant to force is a physical prior that is asserted rather than validated, and Appendix C.1's larger 1-10 N MAE is consistent with that concern; however, an unvalidated assumption is a correctness/generalization risk, not a circular step, because no predicted quantity equals an input by construction. No circularity found.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

Everything the central claim rests on is either borrowed from SANA and FeelAnyForce or asserted without independent evidence. The largest hidden inputs are the frozen natural-image autoencoder, the trust in FeelAnyForce force labels, and the physically questionable assumption that contact-mask shape is force-invariant. No free parameter is fitted to the downstream results, but several choices (evaluation force range, generated dataset sizes, fixed force 6.5 N) shape the reported gains.

free parameters (4)
  • contact mask manual alignment = 1 px translation / 1 deg rotation
    Each contact mask is manually aligned to the reference image; this human annotation defines the position-control signal and is not automated or shipped.
  • force evaluation range cutoff = 4-10 N, excludes 1-4 N
    Force estimation is evaluated only on 4-10 N because the second stage is less accurate for small forces (Sec. 4.2, App. C.1); this post-hoc boundary limits the force-control claim.
  • fixed force for pose experiments = 6.5 N (median)
    The 'fixed' force condition in Table 2 is set to the dataset median; it is a reference condition chosen by hand, not fitted to the result.
  • generated dataset sizes = 20k/40k force; 30k pose; 2.4k/4.8k classification
    Chosen per task; reported gains hold at these sizes, and App. C.2 shows adding 30k generated to 15k real slightly degrades force MAE.
assumptions (4)
  • domain assumption SANA autoencoder (E, D), pretrained on natural images, provides a latent space that faithfully preserves tactile texture, color, and gel deformation
    The frozen SANA encoder/decoder is used for all tactile images without tactile-specific fine-tuning or reconstruction validation (Sec. 3.1).
  • domain assumption FeelAnyForce 3D force annotations are accurate ground truth
    The force-control generator and the force-estimation benchmark both rely on FeelAnyForce [33] force labels (Sec. 3.1).
  • ad hoc to paper The contact-mask shape is invariant to applied force; position variation is captured by rigid translation or rotation of a mask from the initial-contact reference image
    Stated in Sec. 3.1 (Position-Control Mask); physically, contact-patch area typically grows with force, and the paper does not validate force-invariance of the mask.
  • domain assumption Relative force delta F = F_t minus F_i is a sufficient conditioning signal to transfer deformation and texture from the reference to the target force
    The generator is trained to map (reference, delta F) to the target image; this assumes the residual formulation captures the physics across the full force range (Sec. 3.1).
invented entities (1)
  • contact mask (binary, per-object) independent evidence
    purpose: Compact position-control signal marking the approximate contact area; position variation is generated by translating and rotating the mask
    The mask is validated through pose-estimation experiments on real tactile test images, including a self-collected unseen T-shape, which gives a falsifiable handle; however, masks are manually aligned and no mask-generation algorithm is released.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ControlTac: Force- and Position-Controlled Tactile Data Augmentation with a Single Reference Image." pith.science (2026). https://pith.science/paper/7ODSNVPB

@misc{pith2026250520498,
  author       = {Pith},
  title        = {Pith review of: ControlTac: Force- and Position-Controlled Tactile Data Augmentation with a Single Reference Image},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7ODSNVPB}},
  note         = {Machine review of arXiv:2505.20498}
}
read the original abstract

Vision-based tactile sensing has been widely used in perception, reconstruction, and robotic manipulation. However, collecting large-scale tactile data remains costly due to the localized nature of sensor-object interactions and inconsistencies across sensor instances. Existing approaches to scaling tactile data, such as simulation and free-form tactile generation, often suffer from unrealistic output and poor transferability to downstream tasks. To address this, we propose ControlTac, a two-stage controllable framework that generates realistic tactile images conditioned on a single reference tactile image, contact force, and contact position. With those physical priors as control input, ControlTac generates physically plausible and varied tactile images that can be used for effective data augmentation. Through experiments on three downstream tasks, we demonstrate that ControlTac can effectively augment tactile datasets and lead to consistent gains. Our three real-world experiments further validate the practical utility of our approach. Project page: https://dongyuluo.github.io/controltac.

Figures

Figures reproduced from arXiv: 2505.20498 by the authors.

Figure 1
Figure 1. Illustrations of CONTROLTAC’s utilities: starting from a single reference image, CON￾TROLTAC can generate tens of thousands augmented tactile images with various contact forces and contact positions (Left). These augmented images can then be used for various downstream tasks (Middle) and deployed in three real-world experiments (Right). Abstract Vision-based tactile sensing has been widely used in perception, recons… view at source ↗
Figure 2
Figure 2. Comparison of tactile data generation approaches. We evaluate whether each method produces visually realistic images, generates varied outputs from a single input (rather than collapsing to a mean image), and allows control via physical inputs. We compare CONTROLTAC with three other directions: Text2Tactile [14, 22], Visual2Tactile [23, 16, 15], and Simulation [11, 12, 13]. 1 Introduction Vision-based tactile sensin… view at source ↗
Figure 3
Figure 3. Illustration of our controllable tactile generation framework. a) The Force-Control component of CONTROLTAC. We input the tactile image x without background B into the DiT, which is conditioned on the 3D force ∆F. b) The Position-Control component of CONTROLTAC. We copy the DiT from the first stage and finetune it with ControlNet conditioned on the contact mask c for generating realistic tactile image yB conditioned… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Then, the variations of contact position are captured by translating and rotating the mask. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 4
Figure 4. Figure 4: Qualitative Generation Results. The first column displays 3D previews of six objects, followed by the input tactile image (Ref. Image) in the second column and the Contact Mask in the third column. The fourth column shows the initial force (top) and target force (botto…
Figure 5
Figure 5. Figure 5: Force estimation performance (MAE) across different quantities of real and generated data. The normal force range is 1–10 N. After validating the force-control generator in CONTROLTAC, we evaluate the full framework. We find that after training the second stage with Co…
Figure 6
Figure 6. Figure 6: Force estimation performance (MAE) with different data. The sample sizes of 750, 3,750, 7,500, 11,250, and 15,000 are shown as 1, 1/4, 2/4, 3/4, and 4/4 of the contact angles. the real tactile dataset, each object only has 300 unique contact positions, and we randomly …
Figure 7
Figure 7. Figure 7: Qualitative examples of real-world experiments. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Visualization of various angles. Note: The rotational symmetry of spheres renders their angular representations redundant, and thus they are not included here. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Error map comparison between CONTROLTAC and two baseline models. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: Generated tactile images using the force-control generation component of [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: Six objects and their corresponding tactile images used in the classification task. [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation

    cs.RO 2026-07 conditional novelty 6.0 of 10

    FELT predicts finger pressure maps from RGB images and uses them or their learned features to improve manipulation policies without real tactile sensors at deployment.

  2. OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A policy-agnostic two-stage real-world RL method learns tactile residual corrections on frozen visual policies, lifting contact-rich task success from 5–40% to 85–100% in under 80 minutes.

Reference graph

Works this paper leans on

77 extracted references · 52 canonical work pages · cited by 2 Pith papers

  1. [1]

    Rui Li and Edward H. Adelson. Sensing and recognizing surface textures using a gelsight sensor. In2013 IEEE Conference on Computer Vision and Pattern Recognition, pages 1241–1247, 2013

    work page 2013

  2. [2]

    The objectfolder benchmark: Multisensory learning with neural and real objects

    Ruohan Gao, Yiming Dou, Hao Li, Tanmay Agarwal, Jeannette Bohg, Yunzhu Li, Li Fei-Fei, and Jiajun Wu. The objectfolder benchmark: Multisensory learning with neural and real objects. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17276–17286, June 2023

    work page 2023

  3. [3]

    Normalflow: Fast, robust, and accurate contact-based object 6dof pose tracking with vision-based tactile sensors.IEEE Robotics and Automation Letters, pages 1–8, 2024

    Hung-Jui Huang, Michael Kaess, and Wenzhen Yuan. Normalflow: Fast, robust, and accurate contact-based object 6dof pose tracking with vision-based tactile sensors.IEEE Robotics and Automation Letters, pages 1–8, 2024

    work page 2024

  4. [4]

    Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation

    Sudharshan Suresh, Haozhi Qi, Tingfan Wu, Taosha Fan, Luis Pineda, Mike Lambeta, Jitendra Malik, Mrinal Kalakrishnan, Roberto Calandra, Michael Kaess, Joseph Ortiz, and Mustafa Mukadam. Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation. Science Robotics, page adl0628, 2024

    work page 2024

  5. [5]

    Touch-gs: Visual-tactile supervised 3d gaussian splatting

    Aiden Swann, Matthew Strong, Won Kyung Do, Gadiel Sznaier Camps, Mac Schwager, and Monroe Kennedy. Touch-gs: Visual-tactile supervised 3d gaussian splatting. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 10511–10518, 2024

    work page 2024

  6. [6]

    Jha, Diego Romeres, Sangwoon Kim, Daniel Nikovski, and Alberto Rodriguez

    Siyuan Dong, Devesh K. Jha, Diego Romeres, Sangwoon Kim, Daniel Nikovski, and Alberto Rodriguez. Tactile-rl for insertion: Generalization to objects of unknown geometry, 2021

    work page 2021

  7. [7]

    Lee, Huazhe Xu, Edward Adelson, Li Fei-Fei, Ruohan Gao, and Jiajun Wu

    Hao Li, Yizhi Zhang, Junzhe Zhu, Shaoxiong Wang, Michelle A. Lee, Huazhe Xu, Edward Adelson, Li Fei-Fei, Ruohan Gao, and Jiajun Wu. See, hear, and feel: Smart sensory fusion for robotic manipulation. InCoRL, 2022

    work page 2022

  8. [8]

    Mimictouch: Leveraging multi-modal human tactile demonstrations for contact-rich manipulation

    Kelin Yu, Yunhai Han, Qixian Wang, Vaibhav Saxena, Danfei Xu, and Ye Zhao. Mimictouch: Leveraging multi-modal human tactile demonstrations for contact-rich manipulation. In8th Annual Conference on Robot Learning, 2024

    work page 2024

Show all 77 references
  1. [9]

    The impact of data augmentation on tactile-based object classification using deep learning approach

    Philip Maus, Jaeseok Kim, Olivia Nocentini, Muhammad Zain Bashir, and Filippo Cavallo. The impact of data augmentation on tactile-based object classification using deep learning approach. IEEE Sensors Journal, 22(14):14574–14583, 2022

    2022

  2. [10]

    Geometric transformation: Tactile data augmentation for robotic learning

    Gang Yan, Jun Yuyeol, Satoshi Funabashi, Tito Pradhono Tomo, Sophon Somlor, Alexander Schmitz, and Shigeki Sugano. Geometric transformation: Tactile data augmentation for robotic learning. In2023 IEEE International Conference on Development and Learning (ICDL), pages 346–353. ...

    2023

  3. [11]

    Tacto: A fast, flexible, and open-source simulator for high-resolution vision-based tactile sensors.IEEE Robotics and Automation Letters, 7(2):3930–3937, 2022

    Shaoxiong Wang, Mike Lambeta, Po-Wei Chou, and Roberto Calandra. Tacto: A fast, flexible, and open-source simulator for high-resolution vision-based tactile sensors.IEEE Robotics and Automation Letters, 7(2):3930–3937, 2022

    2022

  4. [12]

    Taxim: An example-based simulation model for gelsight tactile sensors.IEEE Robotics and Automation Letters, 7(2):2361–2368, 2022

    Zilin Si and Wenzhen Yuan. Taxim: An example-based simulation model for gelsight tactile sensors.IEEE Robotics and Automation Letters, 7(2):2361–2368, 2022

    2022

  5. [13]

    DIFFTACTILE: A physics-based differentiable tactile simulator for contact-rich robotic manipulation

    Zilin Si, Gu Zhang, Qingwei Ben, Branden Romero, Zhou Xian, Chao Liu, and Chuang Gan. DIFFTACTILE: A physics-based differentiable tactile simulator for contact-rich robotic manipulation. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  6. [14]

    Texttoucher: Fine-grained text-to-touch generation

    Jiahang Tu, Hao Fu, Fengyu Yang, Hanbin Zhao, Chao Zhang, and Hui Qian. Texttoucher: Fine-grained text-to-touch generation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 7455–7463, 2025

    2025

  7. [15]

    Touching a neRF: Leveraging neural radiance fields for tactile sensory data generation

    Shaohong Zhong, Alessandro Albini, Oiwi Parker Jones, Perla Maiolino, and Ingmar Posner. Touching a neRF: Leveraging neural radiance fields for tactile sensory data generation. In6th Annual Conference on Robot Learning, 2022. 11

    2022

  8. [16]

    Tactile-augmented radiance fields.arXiv preprint arXiv:2405.04534, 2024

    Yiming Dou, Fengyu Yang, Yi Liu, Antonio Loquercio, and Andrew Owens. Tactile-augmented radiance fields.arXiv preprint arXiv:2405.04534, 2024

    2024 arXiv

  9. [17]

    Touch and go: Learning from human-collected vision and touch

    Fengyu Yang, Chenyang Ma, Jiacheng Zhang, Jing Zhu, Wenzhen Yuan, and Andrew Owens. Touch and go: Learning from human-collected vision and touch. InThirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022

    2022

  10. [18]

    Jialiang Zhao, Yuxiang Ma, Lirui Wang, and Edward H. Adelson. Transferable tactile trans- formers for representation learning across diverse sensors and tasks, 2024

    2024

  11. [19]

    Sparsh: Self-supervised touch representations for vision-based tactile sensing

    Carolina Higuera, Akash Sharma, Chaithanya Krishna Bodduluri, Taosha Fan, Patrick Lancaster, Mrinal Kalakrishnan, Michael Kaess, Byron Boots, Mike Lambeta, Tingfan Wu, and Mustafa Mukadam. Sparsh: Self-supervised touch representations for vision-based tactile sensing. In 8th A...

    2024

  12. [20]

    Sensor-invariant tactile repre- sentation

    Harsh Gupta, Yuchen Mo, Shengmiao Jin, and Wenzhen Yuan. Sensor-invariant tactile repre- sentation. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  13. [21]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023

    2023

  14. [22]

    Binding touch to everything: Learning unified multimodal tactile representations, 2024

    Fengyu Yang, Chao Feng, Ziyang Chen, Hyoungseob Park, Daniel Wang, Yiming Dou, Ziyao Zeng, Xien Chen, Rit Gangopadhyay, Andrew Owens, and Alex Wong. Binding touch to everything: Learning unified multimodal tactile representations, 2024

    2024

  15. [23]

    Connecting touch and vision via cross-modal prediction

    Yunzhu Li, Jun-Yan Zhu, Russ Tedrake, and Antonio Torralba. Connecting touch and vision via cross-modal prediction. InThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019

    2019

  16. [24]

    Wenzhen Yuan, Siyuan Dong, and Edward H. Adelson. Gelsight: High-resolution robot tactile sensors for estimating geometry and force.Sensors, 17(12), 2017

    2017

  17. [25]

    Gelslim3.0: High-resolution measurement of shape, force and slip in a compact tactile-sensing finger, 2021

    Ian Taylor, Siyuan Dong, and Alberto Rodriguez. Gelslim3.0: High-resolution measurement of shape, force and slip in a compact tactile-sensing finger, 2021

    2021

  18. [26]

    Digit: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation.IEEE Robotics and Automation Letters, 5(3):3838–3845, July 2020

    Mike Lambeta, Po-Wei Chou, Stephen Tian, Brian Yang, Benjamin Maloon, Victoria Rose Most, Dave Stroud, Raymond Santos, Ahmad Byagowi, Gregg Kammerer, Dinesh Jayaraman, and Roberto Calandra. Digit: A novel design for a low-cost compact high-resolution tactile sensor with applic...

    2020

  19. [27]

    9dtact: A compact vision-based tactile sensor for accurate 3d shape reconstruction and generalizable 6d force estimation.arXiv preprint arXiv:2308.14277, 2023

    Changyi Lin, Han Zhang, Jikai Xu, Lei Wu, and Huazhe Xu. 9dtact: A compact vision-based tactile sensor for accurate 3d shape reconstruction and generalizable 6d force estimation.arXiv preprint arXiv:2308.14277, 2023

    2023 arXiv

  20. [28]

    Reskin:versatile, replaceable, lasting tactile skins

    Raunaq Bhirangi, Tess Hellebrekers, Carmel Majidi, and Abhinav Gupta. Reskin:versatile, replaceable, lasting tactile skins. InCoRL, 2021

    2021

  21. [29]

    Anyskin: Plug-and-play skin sensing for robotic touch, 2024

    Raunaq Bhirangi, Venkatesh Pattabiraman, Enes Erciyes, Yifeng Cao, Tess Hellebrekers, and Lerrel Pinto. Anyskin: Plug-and-play skin sensing for robotic touch, 2024

    2024

  22. [30]

    Learning the signatures of the human grasp using a scalable tactile glove

    Subramanian Sundaram, Petr Kellnhofer, Yunzhu Li, Jun-Yan Zhu, Antonio Torralba, and Wojciech Matusik. Learning the signatures of the human grasp using a scalable tactile glove. Nature, 569(7758), 2019

    2019

  23. [31]

    3d vitac:learning fine-grained manipulation with visuo-tactile sensing

    Binghao Huang, Yixuan Wang, Xinyi Yang, Yiyue Luo, and Yunzhu Li. 3d vitac:learning fine-grained manipulation with visuo-tactile sensing. InProceedings of Robotics: Conference on Robot Learning(CoRL), 2024

    2024

  24. [32]

    Dense tactile force distribution estimation using gelslim and inverse fem, 2019

    Daolin Ma, Elliott Donlon, Siyuan Dong, and Alberto Rodriguez. Dense tactile force distribution estimation using gelslim and inverse fem, 2019

    2019

  25. [33]

    Feelanyforce: Estimating contact force feedback from tactile sensation for vision-based tactile sensors, 2024

    Amir-Hossein Shahidzadeh, Gabriele Caddeo, Koushik Alapati, Lorenzo Natale, Cornelia Fermuller, and Yiannis Aloimonos. Feelanyforce: Estimating contact force feedback from tactile sensation for vision-based tactile sensors, 2024. 12

    2024

  26. [34]

    Understanding dynamic tactile sensing for liquid property estimation, 2022

    Hung-Jui Huang, Xiaofeng Guo, and Wenzhen Yuan. Understanding dynamic tactile sensing for liquid property estimation, 2022

    2022

  27. [35]

    Srinivasan, and Edward H

    Wenzhen Yuan, Chenzhuo Zhu, Andrew Owens, Mandayam A. Srinivasan, and Edward H. Adelson. Shape-independent hardness estimation using deep learning and a gelsight tactile sensor. In2017 IEEE International Conference on Robotics and Automation (ICRA), page 951–958. IEEE, May 2017

    2017

  28. [36]

    Tactile dreamfu- sion: Exploiting tactile sensing for 3d generation, 2024

    Ruihan Gao, Kangle Deng, Gengshan Yang, Wenzhen Yuan, and Jun-Yan Zhu. Tactile dreamfu- sion: Exploiting tactile sensing for 3d generation, 2024

    2024

  29. [37]

    Actexplore: Active tactile exploration on unknown objects

    Amir-Hossein Shahidzadeh, Seong Jong Yoo, Pavan Mantripragada, Chahat Deep Singh, Cor- nelia Fermüller, and Yiannis Aloimonos. Actexplore: Active tactile exploration on unknown objects. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 3411–3418, May 2024

    2024

  30. [38]

    Slip detection with combined tactile and visual information

    Jianhua Li, Siyuan Dong, and Edward Adelson. Slip detection with combined tactile and visual information. In2018 IEEE International Conference on Robotics and Automation (ICRA), pages 7772–7777, 2018

    2018

  31. [39]

    Adelson, and Sergey Levine

    Roberto Calandra, Andrew Owens, Dinesh Jayaraman, Justin Lin, Wenzhen Yuan, Jitendra Malik, Edward H. Adelson, and Sergey Levine. More than a feeling: Learning to grasp and regrasp using vision and touch.IEEE Robotics and Automation Letters, 3(4):3300–3307, October 2018

    2018

  32. [40]

    Adelson, and Sergey Levine

    Roberto Calandra, Andrew Owens, Manu Upadhyaya, Wenzhen Yuan, Justin Lin, Edward H. Adelson, and Sergey Levine. The feeling of success: Does touch sensing help predict grasp outcomes?, 2025

    2025

  33. [41]

    Learning generalizable vision-tactile robotic grasping strategy for deformable objects via transformer.IEEE/ASME Transactions on Mechatronics, 30(1):554–566, 2025

    Yunhai Han, Kelin Yu, Rahul Batra, Nathan Boyd, Chaitanya Mehta, Tuo Zhao, Yu She, Seth Hutchinson, and Ye Zhao. Learning generalizable vision-tactile robotic grasping strategy for deformable objects via transformer.IEEE/ASME Transactions on Mechatronics, 30(1):554–566, 2025

    2025

  34. [42]

    General in-hand object rotation with vision and touch

    Haozhi Qi, Brent Yi, Sudharshan Suresh, Mike Lambeta, Yi Ma, Roberto Calandra, and Jitendra Malik. General in-hand object rotation with vision and touch. In7th Annual Conference on Robot Learning, 2023

    2023

  35. [43]

    Tactile-based insertion for dense box-packing, 2019

    Siyuan Dong and Alberto Rodriguez. Tactile-based insertion for dense box-packing, 2019

    2019

  36. [44]

    Robopack: Learning tactile-informed dynamics models for dense packing, 2024

    Bo Ai, Stephen Tian, Haochen Shi, Yixuan Wang, Cheston Tan, Yunzhu Li, and Jiajun Wu. Robopack: Learning tactile-informed dynamics models for dense packing, 2024

    2024

  37. [45]

    Objectfolder: A dataset of objects with implicit visual, auditory, and tactile representations

    Ruohan Gao, Yen-Yu Chang, Shivani Mall, Li Fei-Fei, and Jiajun Wu. Objectfolder: A dataset of objects with implicit visual, auditory, and tactile representations. InCoRL, 2021

    2021

  38. [46]

    Objectfolder 2.0: A multisensory object dataset for sim2real transfer

    Ruohan Gao*, Zilin Si*, Yen-Yu Chang*, Samuel Clarke, Jeannette Bohg, Li Fei-Fei, Wenzhen Yuan, and Jiajun Wu. Objectfolder 2.0: A multisensory object dataset for sim2real transfer. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  39. [47]

    Simulation of vision-based tactile sensors using physics based rendering, 2021

    Arpit Agarwal, Tim Man, and Wenzhen Yuan. Simulation of vision-based tactile sensors using physics based rendering, 2021

    2021

  40. [48]

    Anytouch: Learning unified static-dynamic representation across multiple visuo-tactile sensors

    Ruoxuan Feng, Jiangyu Hu, Wenke Xia, TianciGao, Ao Shen, Yuhao Sun, Bin Fang, and Di Hu. Anytouch: Learning unified static-dynamic representation across multiple visuo-tactile sensors. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  41. [49]

    Learning to read braille: Bridging the tactile reality gap with diffusion models, 2023

    Carolina Higuera, Byron Boots, and Mustafa Mukadam. Learning to read braille: Bridging the tactile reality gap with diffusion models, 2023

    2023

  42. [50]

    Generating visual scenes from touch

    Fengyu Yang, Jiacheng Zhang, and Andrew Owens. Generating visual scenes from touch. International Conference on Computer Vision (ICCV), 2023

    2023

  43. [51]

    Touch2touch: Cross-modal tactile generation for object manipulation, 2024

    Samanta Rodriguez, Yiming Dou, Miquel Oller, Andrew Owens, and Nima Fazeli. Touch2touch: Cross-modal tactile generation for object manipulation, 2024. 13

    2024

  44. [52]

    Generative adversarial text to image synthesis

    Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. InInternational conference on machine learning, pages 1060–1069. PMLR, 2016

    2016

  45. [53]

    Image-to-image translation with conditional adversarial networks

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1125–1134, 2017

    2017

  46. [54]

    Unpaired image-to-image translation using cycle-consistent adversarial networks

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. InProceedings of the IEEE international conference on computer vision, pages 2223–2232, 2017

    2017

  47. [55]

    Cvae-gan: fine-grained image generation through asymmetric training

    Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, and Gang Hua. Cvae-gan: fine-grained image generation through asymmetric training. InProceedings of the IEEE international conference on computer vision, pages 2745–2754, 2017

    2017

  48. [56]

    Disentangled representation learning gan for pose- invariant face recognition

    Luan Tran, Xi Yin, and Xiaoming Liu. Disentangled representation learning gan for pose- invariant face recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1415–1424, 2017

    2017

  49. [57]

    Attribute2image: Conditional image generation from visual attributes

    Xinchen Yan, Jimei Yang, Kihyuk Sohn, and Honglak Lee. Attribute2image: Conditional image generation from visual attributes. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 776–791. Springer, 2016

    2016

  50. [58]

    Conditional generative adversarial nets.arXiv preprint arXiv:1411.1784, 2014

    Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets.arXiv preprint arXiv:1411.1784, 2014

    2014 arXiv

  51. [59]

    Learning structured output representation using deep conditional generative models.Advances in neural information processing systems, 28, 2015

    Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models.Advances in neural information processing systems, 28, 2015

    2015

  52. [60]

    Progressive growing of gans for improved quality, stability, and variation.arXiv preprint arXiv:1710.10196, 2017

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation.arXiv preprint arXiv:1710.10196, 2017

    2017 arXiv

  53. [61]

    Understanding the limitations of conditional generative models.arXiv preprint arXiv:1906.01171, 2019

    Ethan Fetaya, Jörn-Henrik Jacobsen, Will Grathwohl, and Richard Zemel. Understanding the limitations of conditional generative models.arXiv preprint arXiv:1906.01171, 2019

    1906 arXiv

  54. [62]

    A survey on generative adversarial networks: Variants, applications, and training.ACM Computing Surveys (CSUR), 54(8):1–49, 2021

    Abdul Jabbar, Xi Li, and Bourahla Omar. A survey on generative adversarial networks: Variants, applications, and training.ACM Computing Surveys (CSUR), 54(8):1–49, 2021

    2021

  55. [63]

    Deep generative models: Survey

    Achraf Oussidi and Azeddine Elhassouny. Deep generative models: Survey. In2018 In- ternational conference on intelligent systems and computer vision (ISCV), pages 1–8. IEEE, 2018

    2018

  56. [64]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  57. [65]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  58. [66]

    Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  59. [67]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  60. [68]

    Pixart- {\delta}: Fast and controllable image generation with latent consistency models

    Junsong Chen, Yue Wu, Simian Luo, Enze Xie, Sayak Paul, Ping Luo, Hang Zhao, and Zhenguo Li. Pixart- {\delta}: Fast and controllable image generation with latent consistency models. arXiv preprint arXiv:2401.05252, 2024. 14

    2024 arXiv

  61. [69]

    Sana: Efficient high-resolution image synthesis with linear diffusion transformers.arXiv preprint arXiv:2410.10629, 2024

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. Sana: Efficient high-resolution image synthesis with linear diffusion transformers.arXiv preprint arXiv:2410.10629, 2024

    arXiv

  62. [70]

    Diff- fluid: Plain diffusion models are effective predictors of flow dynamics.arXiv preprint arXiv:2409.13665, 2024

    Dongyu Luo, Jianyu Wu, Jing Wang, Hairun Xie, Xiangyu Yue, and Shixiang Tang. Diff- fluid: Plain diffusion models are effective predictors of flow dynamics.arXiv preprint arXiv:2409.13665, 2024

    2024 arXiv

  63. [71]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  64. [72]

    A computational approach to edge detection.IEEE Transactions on pattern analysis and machine intelligence, (6):679–698, 1986

    John Canny. A computational approach to edge detection.IEEE Transactions on pattern analysis and machine intelligence, (6):679–698, 1986

    1986

  65. [73]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:...

    2010 arXiv

  66. [74]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  67. [75]

    Imagenet: A large- scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  68. [76]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  69. [77]

    Pytorch image models

    Ross Wightman. Pytorch image models. https://github.com/huggingface/ pytorch-image-models, 2019. 15 A Appendix: Implementation Details A.1 Training Configuration Both force-control generation component and position-control generation component are trained using the AdamW [76] ...

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.