REVIEW 11 cited by
Digitizing Touch with an Artificial Multimodal Fingertip
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Touch is a crucial sensing modality that provides rich information about object properties and interactions with the physical environment. Humans and robots both benefit from using touch to perceive and interact with the surrounding environment (Johansson and Flanagan, 2009; Li et al., 2020; Calandra et al., 2017). However, no existing systems provide rich, multi-modal digital touch-sensing capabilities through a hemispherical compliant embodiment. Here, we describe several conceptual and technological innovations to improve the digitization of touch. These advances are embodied in an artificial finger-shaped sensor with advanced sensing capabilities. Significantly, this fingertip contains high-resolution sensors (~8.3 million taxels) that respond to omnidirectional touch, capture multi-modal signals, and use on-device artificial intelligence to process the data in real time. Evaluations show that the artificial fingertip can resolve spatial features as small as 7 um, sense normal and shear forces with a resolution of 1.01 mN and 1.27 mN, respectively, perceive vibrations up to 10 kHz, sense heat, and even sense odor. Furthermore, it embeds an on-device AI neural network accelerator that acts as a peripheral nervous system on a robot and mimics the reflex arc found in humans. These results demonstrate the possibility of digitizing touch with superhuman performance. The implications are profound, and we anticipate potential applications in robotics (industrial, medical, agricultural, and consumer-level), virtual reality and telepresence, prosthetics, and e-commerce. Toward digitizing touch at scale, we open-source a modular platform to facilitate future research on the nature of touch.
Forward citations
Cited by 11 Pith papers
-
SpikeATac: A Multimodal Tactile Finger with Taxelized Dynamic Sensing for Dexterous Manipulation
A fingertip with 16-taxel PVDF dynamic sensing plus capacitive static sensing enables fast delicate grasping and, with RLHF fine-tuning, in-hand manipulation of fragile objects.
-
FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation
FELT predicts finger pressure maps from RGB images and uses them or their learned features to improve manipulation policies without real tactile sensors at deployment.
-
DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter
A decoupled multimodal diffusion transformer with a LoRA tactile adapter improves real-world bimanual manipulation success by 21 percentage points over a diffusion-policy baseline; a new 50-hour tactile bimanual datas...
-
3D Cal: An Open-Source Software Library for Depth Reconstruction on Vision-Based Tactile Sensors
3D Cal repurposes a 3D printer as an automated calibration rig and trains a lightweight CNN, TouchNet, to reconstruct depth maps for DIGIT and GelSight Mini.
-
Classification of Vision-Based Tactile Sensors: A Review
A review that proposes a four-type classification of vision-based tactile sensors, dividing them into marker-based versus intensity-based transduction with subtypes and combinations.
-
Tactile Beyond Pixels: Multisensory Touch Representations for Robot Manipulation
Sparsh-X is a transformer trained on about one million unlabeled touch interactions that fuses image, audio, motion, and pressure into representations that boost downstream robot manipulation performance over tactile-...
-
ViTaSCOPE: Visuo-tactile Implicit Representation for In-hand Pose and Extrinsic Contact Estimation
ViTaSCOPE jointly estimates in-hand object pose and extrinsic contact location on an object from vision and tactile shear fields, trained solely in simulation and deployed in the real world.
-
TensorTouch: Calibration of Tactile Sensors for High Resolution Stress Tensor and Deformation for Dexterous Manipulation
TensorTouch converts optical tactile sensor images into dense stress tensor, deformation, and contact force fields using finite-element simulation and a hierarchical vision transformer, and uses these fields for selec...
-
Adaptive Visuo-Tactile Fusion with Predictive Force Attention for Dexterous Manipulation
A force-guided attention module and future-force prediction auxiliary task improve visuo-tactile fusion for dexterous manipulation, reaching 93% average success in real robot trials.
-
The MOTIF Hand: A Robotic Hand for Multimodal Observations with Thermal, Inertial, and Force Sensors
The MOTIF hand adds thermal, inertial, and force sensing to a LEAP hand and demonstrates temperature-aware grasping and mass discrimination from fingertip flicks.
-
Towards Tangible Immersion for Cobot Programming-by-Demonstration: Visual, Tactile and Haptic Interfaces for Mixed-Reality Cobot Automation in Semiconductor Manufacturing
A mixed-reality and haptics-augmented programming-by-demonstration pipeline is proposed to let non-experts program collaborative robots for semiconductor handling through demonstrated primitives, but without evaluation data.
Discussion (0). Continue with ORCID to comment.