Awesome Touch

Owing to my personal interest (I am the author of DTact and 9DTact), this collection may focus on vision-based tactile sensors and learning-based applications.

If you want to show your papers here, please add a new issue and provide the titles and other relative information. Thanks a lot.

Sensors
Applications
Software
Review
Thesis
Products
Open Source
Laboratory

Sensors

Applications

Manipulation/Grasping

Localization and manipulation of small parts using GelSight tactile sensing, Li et al., IROS 2014
Tactile-Based Insertion for Dense Box-Packing, Dong et al., IROS 2019
Maintaining Grasps within Slipping Bounds by Monitoring Incipient Slip, Dong et al., ICRA 2019
Manipulation by feel: Touch-based control with deep predictive models, Tian et al., ICRA 2019
Sensing Shear Forces During Food Manipulation: Resolving the Trade-Off Between Range and Sensitivity, Song et al., ICRA 2019
Building a Library of Tactile Skills Based on FingerVision, Belousov et al., Humanoids 2019
Self-Attention Based Visual-Tactile Fusion Learning for Predicting Grasp Outcomes, Cui et al., RAL 2020
Deep Reinforcement Learning for Tactile Robotics: Learning to Type on a Braille Keyboard, Church et al., RAL 2020
Swingbot: Learning physical features from in-hand tac?tile exploration for dynamic swing-up manipulation, Wang et al., IROS 2020
Tactile Dexterity: Manipulation Primitives with Tactile Feedback, Hogan et al., ICRA 2020
Cable manipulation with a tactile-reactive gripper, She et al., IJRR 2021
A Robust Controller for Stable 3D Pinching Using Tactile Sensing, Psomopoulou et al., RAL 2021
Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry, Dong et al., ICRA 2021
Improving Grasp Stability with Rotation Measurement from Tactile Sensing, Kolamuri et al., IROS 2021
Dynamic Modeling of Hand-Object Interactions via Tactile Sensing, Zhang et al., IROS 2021
SEED: Series Elastic End Effectors in 6D for Visuotactile Tool Use, Suh et al., arxiv 2021
Learning Generalizable Vision-Tactile Robotic Grasping Strategy for Deformable Objects via Transformer, Han et al., arxiv 2021
Meta-Residual Policy Learning: Zero-Trial Robot Skill Adaptation via Knowledge Fusion, Hao et al., RAL 2022
Active Visuo-Tactile Interactive Robotic Perception for Accurate Object Pose Estimation in Dense Clutter, Murali et al., RAL 2022
Autonomous Learning of Page Flipping Movements via Tactile Feedback, Zheng et al., TRO 2022
Where Shall I Touch? Vision-Guided Tactile Poking for Transparent Object Grasping, Jiang et al., Transactions on Mechatronics 2022
Efficient Tactile Simulation with Differentiability for Robotic Manipulation, Xu et al., CoRL 2022
Tactile Pose Estimation and Policy Learning for Unknown Object Manipulation, Kelestemur et al., arxiv 2022
Visuo-Tactile Transformers for Manipulation, Chen et al., arxiv 2022
Grasp Stability Prediction with Sim-to-Real Transfer from Tactile Sensing, Si et al., arxiv 2022
Safely Learning Visuo-Tactile Feedback Policies in Real For Industrial Insertion, Fu et al., arxiv 2022
Manipulation via Membranes: High-Resolution and Highly Deformable Tactile Sensing and Control, Oller et al., arxiv 2022
Learning Self-Supervised Representations from Vision and Touch for Active Sliding Perception of Deformable Surfaces, Kerr et al., arxiv 2022
Active Extrinsic Contact Sensing: Application to General Peg-in-Hole Insertion, Kim et al., ICRA 2022
Task-Driven In-Hand Manipulation of Unknown Objects with Tactile Sensing, Pan et al., arxiv 2022
Visual-tactile Fusion for Transparent Object Grasping in Complex Backgrounds, Li et al., arxiv 2022
Integrating High-Resolution Tactile Sensing into Grasp Stability Prediction, Chumbley et al., CRV 2022
Skill generalization of tubular object manipulation with tactile sensing and Sim2Real learning, Zhao et al., Robotics and Autonomous Systems 2023
Learning height for top-down grasps with the DIGIT sensor, Bernardi et al., ICRA 2023
Deep Functional Predictive Control for Strawberry Cluster Manipulation using Tactile Prediction, Nazari et al., arxiv 2023
Cable Routing and Assembly using Tactile-driven Motion Primitives, Wilson et al., ICRA 2023
Tactile-Filter: Interactive Tactile Perception for Part Mating, Ota et al., arxiv 2023
Learning to Detect Slip through Tactile Measures of the Contact Force Field and its Entropy, Hu et al., arxiv 2023
Visuo-Tactile Sensor Enabled Pneumatic Device Towards Compliant Oropharyngeal Swab Sampling, Li et al., arxiv 2023
Integrated Object Deformation and Contact Patch Estimation from Visuo-Tactile Feedback, Merwe et al., arxiv 2023
Learning-Based Slip Detection for Dexterous Manipulation Using GelStereo Sensing, Cui et al., TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023
Enabling Robot Manipulation of Soft and Rigid Objects with Vision-based Tactile Sensors, Welle et al., arxiv 2023
TacMMs: Tactile Mobile Manipulators for Warehouse Automation, He et al., RAL 2023
Visual-Tactile Learning of Garment Unfolding for Robot-Assisted Dressing, Zhang et al., RAL 2023
Grasp Stability Assessment Through Attention-Guided Cross-Modality Fusion and Transfer Learning, Zhang et al., arxiv 2023
TactileVAD: Geometric Aliasing-Aware Dynamics for High-Resolution Tactile Control, Oller et al., CoRL 2023
Precise Robotic Needle-Threading with Tactile Perception and Reinforcement Learning, Yu et al., CoRL 2023
General In-hand Object Rotation with Vision and Touch, Qi et al., CoRL 2023
Inter-finger Small Object Manipulation with DenseTact Optical Tactile Sensor, Do et al., arxiv 2023
Embedded Object Detection and Mapping in Soft Materials Using Optical Tactile Sensing, Solano-Castellanos et al., arxiv 2023
VisTac Towards a Unified Multi-Modal Sensing Finger for Robotic Manipulation, Athar et al., IEEE Sensors 2023
Laboratory Automation: Precision Insertion with Adaptive Fingers utilizing Contact through Sliding with Tactile-based Pose Estimation, Pai et al., arxiv 2023
AcTExplore: Active Tactile Exploration on Unknown Objects, Shahidzadeh et al., arxiv 2023
MimicTouch: Learning Human's Control Strategy with Multi-Modal Tactile Feedbacks, Yu et al., arxiv 2023
Tactile Active Inference Reinforcement Learning for Efficient Robotic Manipulation Skill Acquisition, Liu et al., arxiv 2023
Evetac: An Event-based Optical Tactile Sensor for Robotic Manipulation, Funk et al., TRO 2023
Pose and shear-based tactile servoing, Lloyd et al., IJRR 2023
GelStereo BioTip: Self-Calibrating Bionic Fingertip Visuotactile Sensor for Robotic Manipulation, Cui et al., TMech 2023
Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation, Suresh et al., arxiv 2023
TEXterity: Tactile Extrinsic deXterity, Bronars et al., arxiv 2024
Tac-Man: Tactile-Informed Prior-Free Manipulation of Articulated Objects, Zhao et al., arxiv 2024
LeTac-MPC: Learning Model Predictive Control for Tactile-reactive Grasping, Xu et al., arxiv 2024
Visuo-Tactile Pretraining for Cable Plugging, George et al., arxiv 2024
Stable Object Placing using Curl and Diff Features of Vision-based Tactile Sensors, Takahashi et al., arxiv 2024

Classification/Recognition

Majority Voting: Material Classification by Tactile Sensing Using Surface Texture, Jamali et al., TRO 2011
Sensing and recognizing surface textures using a gelsight sensor, Li et al., CVPR 2013
Connecting Look and Feel: Associating the visual and tactile properties of physical materials, Yuan et al., CVPR 2017
Shape-independent hardness estimation using deep learning and a GelSight tactile sensor, Yuan et al., ICRA 2017
Vitac: Feature sharing between vision and tactile sensing for cloth texture recognition, Yuan et al., ICRA 2018
Active clothing material perception using tactile sensing and deep learning, Yuan et al., ICRA 2018
Understanding Dynamic Tactile Sensing for Liquid Property Estimation, Huang et al., arxiv 2022
Visuotactile Affordances for Cloth Manipulation with Local Control, Sunil et al., arxiv 2022
See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation, Li et al., arxiv 2022
TANDEM: Learning Joint Exploration and Decision Making With Tactile Sensors, Xu et al., RAL 2022
TANDEM3D: Active Tactile Exploration for 3D Object Recognition, Xu et al., ICRA 2023
Multimodal Zero-Shot Learning for Tactile Texture Recognition, Cao et al., arxiv 2023
Controllable Visual-Tactile Synthesis, Gao et al., arxiv 2023
Estimating Properties of Solid Particles Inside Container Using Touch Sensing, Guo et al., arxiv 2023
Aerial Interaction with Tactile Sensing, Guo et al., arxiv 2023
Sim2Real Bilevel Adaptation for Object Surface Classification using Vision-Based Tactile Sensors, Caddeo et al., arxiv 2023
A Predictive Model for Tactile Force Estimation using Audio-Tactile Data, Niederhauser et al., RAL 2023
A Touch, Vision, and Language Dataset for Multimodal Alignment, Fu et al., arxiv 2024

Mapping/Localization

Tracking objects with point clouds from vision and touch, Izatt et al., ICRA 2017
3D Shape Perception from Monocular Vision, Touch, and Shape Priors, Wang et al., IROS 2018
End-to-end pixelwise surface normal estimation with convolutional neural networks and shape reconstruction using GelSight sensor, Li et al., ROBIO 2018
From Pixels to Percepts: Highly Robust Edge Perception and Contour Following Using Deep Learning and an Optical Biomimetic Tactile Sensor, Lepora et al., RAL 2019
Convolutional Autoencoder for Feature Extraction in Tactile Sensing, Polic et al., RAL 2019
Tactile Mapping and Localization from High-Resolution Tactile Imprints, Bauza et al., ICRA 2019
Fast Model-Based Contact Patch and Pose Estimation for Highly Deformable Dense-Geometry Tactile Sensors, Kuppuswamy et al., RAL 2020
Deep Gated Multi-modal Learning: In-hand Object Pose Changes Estimation using Tactile and Image Data, Anzai et al., IROS 2020
3D Shape Reconstruction from Vision and Touch, Smith et al., NeurIPS 2020
Tactile SLAM: Real-time inference of shape and pose from planar pushing, Suresh et al., ICRA 2021
Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering, Bauza et al., CoRL 2021
Active 3D Shape Reconstruction from Vision and Touch, Smith et al., NeurIPS 2021
Active Visuo-Tactile Point Cloud Registration for Accurate Pose Estimation of Objects in an Unknown Workspace, Murali et al., arxiv 2021
Learning Tactile Models for Factor Graph-based Estimation, Sodhi et al., ICRA 2021
Tac2Pose: Tactile Object Pose Estimation from the First Touch, Bauza et al., arxiv 2022
ShapeMap 3-D: Efficient shape mapping through dense touch and vision, Suresh et al., ICRA 2022
PatchGraph: In-hand tactile tracking with learned surface normals, Sodhi et al., ICRA 2022
MidasTouch: Monte-Carlo inference over distributions across sliding touch, Suresh et al., CoRL 2022
Enhancing Generalizable 6D Pose Tracking of an In-Hand Object with Tactile Sensing, Xu et al., arxiv 2022
Using Collocated Vision and Tactile Sensors for Visual Servoing and Localization, Chaudhury et al., arxiv 2022
Simultaneous Contact Location and Object Pose Estimation Using Proprioception and Tactile Feedback, Sipos et al., arxiv 2022
VisuoTactile 6D Pose Estimation of an In-Hand Object Using Vision and Tactile Sensor Data, Dikhale et al., RAL 2022
In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors, Gao et al., IEEE 2023
Tac2Structure: Object Surface Reconstruction Only Through Multi Times Touch, Lu et al., RAL 2023
Tac-VGNN: A Voronoi Graph Neural Network for Pose-Based Tactile Servoing, Fan et al., arxiv 2023
FingerSLAM: Closed-loop Unknown Object Localization and Reconstruction from Visuo-tactile Feedback, Zhao et al., arxiv 2023
TactoFind: A Tactile Only System for Object Retrieval, Pai et al., arxiv 2023
Touch if it's transparent! ACTOR: Active Tactile-based Category-Level Transparent Object Reconstruction, Murali et al., arxiv 2023
ViHOPE: Visuotactile In-Hand Object 6D Pose Estimation with Shape Completion, Li et al., RAL 2023
Sliding Touch-based Exploration for Modeling Unknown Object Shape with Multi-fingered Hands, Chen et al., IROS 2023
Collision-aware In-hand 6D Object Pose Estimation using Multiple Vision-based Tactile Sensors, Caddeo et al., ICRA 2023
Tactile-Filter: Interactive Tactile Perception for Part Mating, Ota et al., RSS 2023
Enhanced Information Extraction from Cylindrical Visual-Tactile Sensors via Image Fusion, Li et al., arxiv 2023
TouchSDF: A DeepSDF Approach for 3D Shape Reconstruction using Vision-Based Tactile Sensing, Comi et al., arxiv 2023
Grasping, Part Identification, and Pose Refinement in One Shot with a Tactile Gripper, Lim et al., arxiv 2023
Measuring Object Rotation via Visuo-Tactile Segmentation of Grasping Region, Castano-Amoros et al., RAL 2023
TransTouch: Learning Transparent Objects Depth Sensing Through Sparse Touches, Bian et al., IROS 2023
Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces, Comi et al., arxiv 2024

Extrinsic Contact Sensing

Extrinsic Contact Sensing with Relative-Motion Tracking from Distributed Tactile Measurements, Ma et al., ICRA 2021
Simultaneous Tactile Estimation and Control of Extrinsic Contact, Kim et al., ICRA 2023
Neural Contact Fields: Tracking Extrinsic Contact with Tactile Sensing, Higuera et al., ICRA 2023
Perceiving Extrinsic Contacts from Touch Improves Learning Insertion Policies, Higuera et al., arxiv 2023
Simultaneous Tactile Estimation and Control for Extrinsic Dexterity, Bronars et al., arxiv 2023
TEXterity: Tactile Extrinsic deXterity, Bronars et al., arxiv 2024

Image Generation

Deep Tactile Experience: Estimating Tactile Sensor Output from Depth Sensor Data, Patel et al., IROS 2020
Touching a NeRF: Leveraging Neural Radiance Fields for Tactile Sensory Data Generation, Zhong et al., CoRL 2022
Vis2Hap: Vision-based Haptic Rendering by Cross-modal Generation, Cao et al., ICRA 2023
Learning to Read Braille: Bridging the Tactile Reality Gap with Diffusion Models, Higuera et al., ICRA 2023
Generating Visual Scenes from Touch, Yang et al., ICCV 2023

Dexterous Hand Manipulation

Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation, Khandate et al., arxiv 2023
Rotating without Seeing: Towards In-hand Dexterity through Touch, Yin et al., arxiv 2023
Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play, Guzey et al., arxiv 2023
TacGNN: Learning Tactile-based In-hand Manipulation with a Blind Robot using Hierarchical Graph Neural Network, Yang et al., arxiv 2023
General In-Hand Object Rotation with Vision and Touch, Qi et al., arxiv 2023

Others

Exploiting Distributed Tactile Sensors to Drive a Robot Arm Through Obstacles, Albini et al., RAL 2021
Extended Tactile Perception: Vibration Sensing through Tools and Grasped Objects, Taunyazov et al., IROS 2021
MidasTouch: Monte-Carlo inference over distributions across sliding touch, Suresh et al., CoRL 2022
SLURP! Spectroscopy of Liquids Using Robot Pre-Touch Sensing, Hanson et al., arxiv 2022
Learning to Synthesize Volumetric Meshes from Vision-based Tactile Imprints, Zhu et al., arxiv 2022
Going In Blind: Object Motion Classification using Distributed Tactile Sensing for Safe Reaching in Clutter, Thomasson et al., arxiv 2022
Touch and Go: Learning from Human-Collected Vision and Touch, Yang et al., NeurIPS 2022
Development and Evaluation of a Learning-based Model for Real-time Haptic Texture Rendering, Heravi et al., arxiv 2022
Investigating Vision Foundational Models for Tactile Representation Learning, Zandonati et al., arxiv 2023
In-situ Mechanical Calibration for Vision-based Tactile Sensors, Zhao et al., ICRA 2023
Multimodal Visual-Tactile Representation Learning through Self-Supervised Contrastive Pre-Training, Dave et al., arxiv 2024
Low-Cost Teleoperation with Haptic Feedback through Vision-based Tactile Sensors for Rigid and Soft Object Manipulation, Lippi et al., arxiv 2024

Software

Simulator

Generation of GelSight Tactile Images for Sim2Real Learning, Gomes et al., RAL 2021
Simulation of Vision-based Tactile Sensors using Physics based Rendering, Agarwal et al., ICRA 2021
Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation, Church et al., CoRL 2021
Tactile Gym 2.0: Sim-to-Real Deep Reinforcement Learning for Comparing Low-Cost High-Resolution Robot Touch, Lin et al., RAL 2022
TACTO: A Fast, Flexible, and Open-Source Simulator for High-Resolution Vision-Based Tactile Sensors, Wang et al., RAL 2022
Taxim: An Example-Based Simulation Model for GelSight Tactile Sensors, Si et al., RAL 2022
Bidirectional Sim-to-Real Transfer for GelSight Tactile Sensors With CycleGAN, Chen et al., RAL 2022
Learning the Dynamics of Compliant Tool-Environment Interaction for Visuo-Tactile Contact Servoing, Merwe et al., arxiv 2022
Efficient Tactile Simulation with Differentiability for Robotic Manipulation, Xu et al., CoRL 2022
Tacchi: A Pluggable and Low Computational Cost Elastomer Deformation Simulator for Optical Tactile Sensors, Chen et al., RAL 2023
Unsupervised Adversarial Domain Adaptation for Sim-to-Real Transfer of Tactile Images, Jing et al., Transactions on Instrumentation and Measurement 2023
Beyond Flat GelSight Sensors: Simulation of Optical Tactile Sensors of Complex Morphologies for Sim2Real Learning, Gomes et al., RSS 2023
Marker-Embedded Tactile Image Generation via Generative Adversarial Networks, Kim et al., RAL 2023
Augmenting Tactile Simulators with Real-like and Zero-Shot Capabilities, Azulay et al., RAL 2023
Tactile Imprint Simulation of GelStereo Visuotactile Sensors, Cui et al., ICMA 2023
TacIPC: Intersection- and Inversion-free FEM-based Elastomer Simulation For Optical Tactile Sensors, Du et al., arxiv 2023
General-Purpose Sim2Real Protocol for Learning Contact-Rich Manipulation With Marker-Based Visuotactile Sensors, Chen et al., TRO 2024

Library

PyTouch: A Machine Learning Library for Touch Processing, Lambeta et al., ICRA 2021

Review

Artificial Sense of Slip—A Review, Francomano et al., IEEE Sensors 2013
Tactile sensing in dexterous robot hands — Review, Kappassov et al., Robotics and Autonomous Systems 2015
Tactile Image Sensors Employing Camera: A Review, Shimonomura et al., MDPI Sensors 2019
Tactile Sensors for Advanced Intelligent Systems, Wang et al., Advanced Intelligent Systems 2019
Visuotactile Sensors With Emphasis on GelSight Sensor: A Review, C. Abad et al., IEEE Sensors 2020
On the Design and Development of Vision-based Tactile Sensors, Shah et al., JINT 2021
A survey of visuotactile sensing technologies for robotic manipulation, Cui et al., Chinese Journal of Intelligent Science and Technology 2022
Hardware Technology of Vision-Based Tactile Sensor: A Review, Zhang et al., IEEE Sensors 2022

-Marker Displacement MethodUsedinVision-Based Tactile Sensors—From 2D to 3D: A Review, LI et al., IEEE Sensors 2023

Thesis

Master

Tactile Measurement with a GelSight Sensor, Wenzhen Yuan, Master, 2014
Robust object pose estimation with point clouds from vision and touch, Gregory Russell, Master, 2017
3D shape perception from vision and touch, Shaoxiong Wang, Master, 2019
Vision-based proprioception of a soft robotic finger with tactile sensing, Sandra Q. Liu, Master, 2020
Discovering the patterns of human-environment interactions using scalable functional textiles, Yiyue Luo, Master, 2020
Deformable Object Manipulation with a Tactile Reactive Gripper, Neha Sunil, Master, 2021

PhD

Touching is believing : sensing and analyzing touch information with GelSight, Rui Li, Ph.D., 2015
High-resolution tactile sensing for robotic perception, Wenzhen Yuan, Ph.D., 2018
Reactive manipulation with contact models and tactile feedback, Francois R. Hogan, Ph.D., 2020
Data-driven Tactile Sensing using Spatially Overlapping Signals, Pedro Piacenza, Ph.D., 2020
High-resolution Tactile Sensing for Reactive Robotic Manipulation, Siyuan Dong, Ph.D., 2021
A general framework for high-resolution robotic tactile sensing: design, simulation, and learning, Carmelo Sferrazza, Ph.D., 2022
Interactive Touch for Manipulation, Shaoxiong Wang, Ph.D., 2022

chenwh14/Awesome-Touch

Awesome Touch

Table of Contents

Sensors

Vision-Based

Visual-Tactile

Non-Vision-Based

Applications

Manipulation/Grasping

Classification/Recognition

Mapping/Localization

Extrinsic Contact Sensing

Image Generation

Dexterous Hand Manipulation

Others

Software

Simulator

Library

Review

Thesis

Master

PhD

Products

Open-Source

Sensor

Sensor Support

Simulator/Library

Dataset

Algorithms

Laboratory