rmui

Robot Manipulation User Interface (RMUI)

Information

This package is the source code of the following paper:

Miniature Tangible Cube: Concept and Design of Target-Object-Oriented User Interface for Dual-Arm Telemanipulation

Dependency

RoboticMaterials/FA-I-Sensor

Installation & Build

Scipy Installation

pip install scipy==1.2.3

Normal Workspace build

mkdir rmui_ws/src -p
cd rmui_ws/src
wget https://raw.githubusercontent.com/knorth55/rmui/master/fc.rosinstall -o .rosinstall
wstool update -j 2
rosdep install --ignore-src --from-path . -y -r
cd ..
catkin build

Device workspace build

ssh <your device>
mkdir rmui_ws/src
# for device
wget https://raw.githubusercontent.com/knorth55/rmui/master/device.rosinstall.${ROS_DISTRO} -o .rosinstall
wstool update -j 2
rosdep install --ignore-src --from-path . -y -r
cd ..
catkin build

Baxter workspace build

cd rmui_ws/src
wstool merge knorth55/rmui/baxter.rosinstall
wstool update -j 2
rosdep install --ignore-src --from-path . -y -r
cd ..
catkin build

SSH to MTCs

# old one, rev 1
ssh -oHostKeyAlgorithms='ssh-rsa' pi@mtc1
# new one, rev 2.1
ssh -oHostKeyAlgorithms='ssh-rsa' pi@mtc2
# new one, rev 2.1
ssh -oHostKeyAlgorithms='ssh-rsa' pi@mtc3

Demo launch

PR2 + MTC

# single MTC
roslaunch rmui_demos pr2_rmui.launch
# multi MTC
roslaunch rmui_demos pr2_multi_rmui.launc scene_name:=x_aligned_two_jetsonsh

PR2 + dummy MTC

# single MTC
roslaunch rmui_demos pr2_rmui_dummy.launch
# multi MTC
roslaunch rmui_demos pr2_multi_rmui_dummy.launch scene_name:=x_aligned_two_jetsons

Baxter + single MTC

# for real robot
roslaunch jsk_baxter_startup baxter_softhand.launch

# single MTC
roslaunch rmui_demos baxter_rmui.launch
# multi MTC
roslaunch rmui_demos baxter_multi_rmui.launch scene_name:=x_aligned_two_jetsons

Baxter + dummy MTC

# for real robot
roslaunch jsk_baxter_startup baxter_softhand.launch

# single MTC
roslaunch rmui_demos baxter_rmui_dummy.launch
# multi MTC
roslaunch rmui_demos baxter_multi_rmui_dummy.launch scene_name:=x_aligned_two_jetsons

Device setup

SD card backup

Please follow here

cd ~/Downloads
sudo dd bs=4M if=/dev/sde of=20211102_mtc2_melodic.img
sudo chown $USER: 20211102_mtc2_melodic.img

wget https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh
chmod +x pishrink.sh
LANG=en_US.UTF-8 sudo ./pishrink.sh 20211102_mtc2_melodic.img 20211102_mtc2_melodic_shrinked.img
tar czf 20211102_mtc2_melodic.img.tar.gz 20211102_mtc2_melodic.img

SD card restore

Download img file.

tar xzf 20211102_mtc2_melodic.img.tar.gz
sudo dd bs=4M if=20211102_mtc2_melodic_shrinked.img of=/dev/sde

Install LED setup

sudo pip install rpi_ws281x

LED setup

Follow jgarff/rpi_ws281x#spi

additional LED setup for melodic

For melodic, please do the following, too.

sudo apt install sysfsutils
sudo echo devices/system/cpu/cpu0/cpufreq/scaling_min_freq = 1000000 > /etc/sysfs.d/99-cpu-min.conf

RMUI device

Miniature Tangible Cube

rosrun rmui_drivers rmui_node.py

Output

~output/proximities: (force_proximity_ros/ProximityArray)
~output/imu: (sensor_msgs/Imu)

Sensors

VCNL4040: Proxmity sensor

rosrun rmui_drivers vcnl4040_node.py

Output

~output: (force_proximity_ros/ProximityStamped)

BNO055: IMU

rosrun rmui_drivers bno055_node.py

Output

~output: (sensor_msgs/Imu)

Multiple VCNL4040 + PCA9547: Proxmity sensors

rosrun rmui_drivers vcnl4040_multiplexa_node.py

Output

~output: (force_proximity_ros/ProximityArray)

Actuators

WX281x & OSTW3535C1A

rosrun rmui_drivers wx281x_node.py

Input

~input: (rmui_msgs/LED)

Citation

@ARTICLE{9483662,
  author={Kitagawa, Shingo and Hasegawa, Shun and Yamaguchi, Naoya and Okada, Kei and Inaba, Masayuki},
  journal={IEEE Robotics and Automation Letters},
  title={Miniature Tangible Cube: Concept and Design of Target-Object-Oriented User Interface for Dual-Arm Telemanipulation},
  year={2021},
  volume={6},
  number={4},
  pages={6977-6984},
  doi={10.1109/LRA.2021.3096475}
}