Commander cannot get follower position
UeFan opened this issue · 4 comments
I find that the commander cannot know where the follower is by using any of the three actions 'ProgressCheck', 'SelectOid' or 'SearchObject'. In this situation, the commander has no way to provide valid instructions for the follower because it only knows where the target object's location is but not the follower's location.
Driver may use its own pose information. Commander may use any information from the simulator, including commander/follower poses. To extract current pose, you can use get_current_pose(agent_id=1) simulator api where agent id = 0 and 1 for commander and follower respectively.
Will the commander be allowed to use the object states extracted from the simulator?
Since the ProgressCheck only returns the sub-tasks description and success, without using simulator information, the commander doesn't know, for example, if the bread slice is successfully put into the toaster in a task of toasting bread slices. The commander only knows if the bread slice is toasted or not, but if it's not, the commander doesn't know what causes the failure, failing to place the bread slice into the toaster or failing to turn on the toaster). So I think if the commander is allowed to check the object states, the commander will know whether the toaster is on, and whether the bread slice is inside the toaster.
Another thing I find it vague is that if the commander is allow to get a target object mask based on the driver's view from the simulator. Currently, the SelectOid only returns a object mask based on the Object Search camera view.
I got my answers here #7 (comment). Thanks!