Visuo-motor learning of simulated iCub robot [1] using a deep learning model implemented in Tensorflow (tested on simple toy task). Github page
Trained checkpoint file is too large to upload. You need to train using the train_rnn.py
or we might upload the file in other places later.
- C++
- Yarp and iCub (iCub_SIM)
- iCub software installation from source
- NOTE: in order to manipulate cylinder objects in the simulator, cylinder-cylinder collision is required because the robot's finger is also a cylinder. You need to install ODE with
--enable-libccd
option. intalling ODE - Screen inside the simulator needs to be ON (It is set to show just checker board image in this sample experiment, but we are going to use it for gesture recognition task later). In order to turn on the screen, you need to change
screen
toon
iniCub_parts_activation.ini
inside{iCub-folder}/contexts/simConfig
directory. - In order to make the manipulation easier, we change the friction between objects and hands. We set the friction coefficient very large (1000000.0). You can change
contactFrictionCoefficient
inode_params.ini
inside{iCub-folder}/contexts/simConfig
directory. - You can find more information about iCub simulatior in simulator README
- Yarp python biding
- Yarp python binding
- iCub-python binding might not be required to run this codes
- Python 2.7
- Tensorflow
- tcmalloc (about memory performance using Tensorflow, refer to Github issues #3009 and Stackoverflow)
- Ubuntu 14.04
- Yarp 2.3.66
- iCub 1.4.0
- Python 2.7 (Anaconda)
- Tensorflow 0.12.x
- We tested the program in a very simple object manipulation task. There is only one object (which can be different location, shapes and sizes) in front of the robot and the robot is trained to grasp the object. The experiment settings are similar to previous works [2] and [3]
- [NOTE] Since the purpose of this code is to test connecting iCub simulator and tensorflow, we ignore some necessary components for training and examining deep neural networks. First, we don't use any validation data although checking validation loss was implemented in the training code (we use same training data to validation). Second, we don't test generalization but only check whether the network do operate as it trained. The testing situation on the simulator is exactly same as the training data. We may conduct real research experiments based on this code in the future.
- Training Dataset
- Visual input: 48x64 size, greyscale value range from [-1,1]
- Joint output: 10 DoF joints from iCub left arm and neck are used and each joint is sparsely transformed into 10 dimension. In conclusion, model's output dimension is 10x10
- Training datasets are collected using a dedicated program which is not shared in this repository. The structure of the dataset program is basically same as the online testing program in this repository but the motor generation part is replaced by hand coded program and saving vision(from iCub camera) and motor(from iCub joint) sequences parts are added. You can either make the data collecting program or change the online testing program depending on your task senario.
- We trained a CNN-RNN structure especially CNN-LSTM model for testing (the model can be changed to VMDNN [2] using
BasicConvCTNNCell
inBasicConvLSTMCell.py
andBasicCTRNNCell
inrnn_cell_ext.py
)
Figure. VMDNN model without attention control. The model in the program is implemented based on the model in the figure. The model receives sequence of visual input and outputs sequence of robot's joint trajectory through time. The figure is adopted from the final report of "Study on Application of Curriculum Learning in Deep Developmental Robotics" project funded by Venture Research Program for Graduate and PhD Students 2016 (VRPGP 2016)
-
For training a model
train_rnn.py
: main program for training a deep neural network model using Tensorflow. Import model frommodel.py
model.py
: defining Tensorflow graph of the network model. you can define a new model you want to test.BasicConvLSTMCell.py
: implementation of convolutional LSTM(or RNN) [4] by loliverhennigh. We addedBasicConvCTRNNCell
where LSTM cells are replaced by continuous time RNN (CTRNN) cells andBasicConvCTNNCell
where LSTM cells are replaced by leaky integrator cells (as in MSTNN model [5])rnn_cell_ext.py
: implementation of normal CTRNNdata
directory: contains datasetdataset_np
directory:npz
format file which contains dataset in list ofnumpy
arraysgestureVideos
directory: contains images to project in screen inside simulatorlabel
directory: sparsely transformed motor targetTRIAL_LIST
directory: list of visuo-motor sequence for training (in the sample experiment, only 4 digits number in the first colunm has meaning. The number of rows are same as the number of training sequences)- 1st digit: object type (1: Ball, 2: Box, 3: Cylinder)
- 2nd digit: object size (1: Big, 2: Small)
- 3rd digit: object location (0-9 : different positions in front of the robot)
- 4th digit: object orientation (set 0 for all data)
vision
directory: visual input
-
For online testing in the simulator
controller
: main program which controls iCub robot and synchronizes other sub-programs for the online testing. YARP is used to communicate between each sub-programs.worldManipulator
: a program which set the task environment according to the direction fromcontroller
(making and removing objects and tables)fingerGrasper
: a program which controls the grasping level (a single scalar value from 0 to 10) as used in previous works [2] and [3]vision
: a program which captures visual scene from iCub camera for the network input and sends it to themain.py
according to the direction fromcontroller
screenWriter
: a program which projects images to the screen inside the simulatormain.py
: a Tensorflow program which loads trained the checkpoint file, calculate model outputs and send it to thecontroller
at each time stephomePositions
directory: contains iCub's initial positionssoftmaxConfig
directory: contains information for motor transform and inverse transformtrialConf
directory: list of test trials
- Run
bash run_programs.sh
(orpython train_rnn.py
with proper options)
- Some options
data_dir
anddata_fn
: for specifying dataset location and file namelog_dir
: a directory for saving Tensorflow checkpoint filedevice
: select CPU or GPU for training- Some hyper-parameters for the model (e.g.
lr
,batch_size
, and etc)
- Launch the simulator
- in one terminal:
yarpserver
- in another terminal:
iCub_SIM
- Build online testing program
- move to
onlineTestingProgram
folder and then move tobuild
folder (make if there isn't) cmake ../
make
- Run online testing program
- Run
python main.py
inonlineTestingProgram
folder (you might need to provide proper options e.g. `--log_dir ./../log_dir01_01')- Some options
log dir
: a directory which contains check point file of trained modelmax_leng
: maximum time steps you want to testuse_data_vision
,use_data_motor
, anduse_data_vision
: for checking offline trained performance and debuggingsave_states
andsave_dir
: for saving internal states of the model while testing.
- Some options
- Run
./worldManipulator
,./fingerGrashper
,./vision
,./screenWriter
and then./controller
in seperate terminals (./controller
should be the last one)- [NOTE] you can make a simple script to run all programs in one terminal (refer to
llauncher
andkiller
inonlineTestingProgram
folder)
- [NOTE] you can make a simple script to run all programs in one terminal (refer to
[1] Tikhanoff, Vadim, et al. "An open-source simulator for cognitive robotics research: the prototype of the iCub humanoid robot simulator." Proceedings of the 8th workshop on performance metrics for intelligent systems. ACM, 2008.
[2] J. Hwang, M. Jung and J. Tani: “A Deep Learning Approach for Seamless Integration of Cognitive Skills for Humanoid Robots”, ICDL International Conference Developmental Learning and Epigenetic Robotics (ICDL-EPIROB), 2016
[3] J. Hwang, M. Jung, M. Madapana, J. Kim, M. Choi, J. Tani: "Achieving “Synergy” in Cognitive Behavior of Humanoids via Deep Learning of Dynamic Visuo-Motor-Attentional Coordination", To appear in Proc. of 2015 IEEE-RAS International Conference on Humanoid Robots
[4] Xingjian, S. H. I., et al. "Convolutional LSTM network: A machine learning approach for precipitation nowcasting." Advances in Neural Information Processing Systems. 2015.
[5] Jung, Minju, Jungsik Hwang, and Jun Tani. "Self-organization of spatio-temporal hierarchy via learning of dynamic visual image patterns on action sequences." PloS one 10.7 (2015): e0131214.