Usage of different grippers and robots?
Closed this issue · 1 comments
is it possible to use different gripper apart from the one shown in demo videos?
Can we use different robots or only the ones used for training?
Is it possible to replace the iphone for any camera that streams the video data alone? as in robotics cost is important and cannot use iphone for all use cases?
-
The gripper we use is one thing we’re constrained to, as it’s the same one that’s used in all of our encoder pretraining and model training data. We’ve experimented evaluating with different tips (black suction ones), but our policies failed to generalize to them.
-
We actually don’t use the robot for data collection, only our stick! That means we can use any 6DOF robot with the same end-effector. We added it onto a UFactory xArm 7 ourselves, and release a mount for it, but it can be added onto just about any robotic arm with just the gripper (we take it off of a reacher grabber like this), a Dynamixel starter set and $10 in additional hardware and 3d-printed parts. Once this is added, just IK needs to be done, accounting for the camera’s relative position to the robot’s original end-effector.
-
Yes! We touch on this briefly in the appendix of our paper (section A.6). As long as the camera’s position is at a similar height and angle as the iPhone’s (75º), you can then match the camera’s FOV more closely using a simple affine transformation that matches the red gripper tip position with the demos contained in our dataset. We, and other Hello Robot Stretch 3 users, have been able to successfully deploy policies using the native RealSense D405 that comes standard on its wrist. There are some additional minor details covered in the appendix section.