Express knowledge about other agent using mirror neuron system
Closed this issue · 8 comments
Imperial will use their mirror neuron / perspective taking system to use the knowledge we acquired for the self (bodypart naming) when expressing our knowledge in terms of the partners body. As simple example, if the iCub learnt about his left arm, he will be able to point at the human's left arm saying its his left arm (this example is not very exciting, but I guess you get the idea)
As a first step, I will implement the above mentioned pointing during the integration meeting at BCBT.
@hjchang @maxime-petit: Let's have a chat on how exactly this should be done next week.
@maxime-petit and @hjchang have been working on this recently, and did good progress. We're now able to get the correspondences not only between selfimage-otherimage (year 2 demo), but also between selfjoint-selfimage, which allows us to get the selfjoint-otherimage (and vice versa). We're currently agreeing on a good format to store these correspondences. Also, we have to think about how to best demonstrate this in the year 3 demo.
Just an update: We're still on this, and hope to have something by the end of the project.
This is mostly done. As soon as @pattacini will have finished the work on the pointing to 3D space (#194), we'll be able to merge the work by @jypuigbo in https://github.com/robotology/wysiwyd/tree/feature/sensationPointBodypart - and then it's just a matter of changing the iCubClient to either call the ARE->point() or the Pattacini->point() depending on whether it is an Object or a Bodypart. Also, we'll need to update the speech but this is also straightforward.
@clement-moulin-frier proposed the following:
First, we do babbling with the hand instead of the finger as we used to do. Then, we show the kinematic structure correspondence on the screen afterwards. Finally, we point to the hand of the human.
@martz21 and @hjchang will check whether the babbling is working fine for the hand. I will take care of the rest.
See a00dd77 - it needs to be tested, in case it works I think we can close this issue.
There was a bug which is fixed in cdaf511 which prohibited this to run properly. Should work now.
Working fine, closing.