How to convert Agent state in Ithor to the same format as in Robothor?

[zhangfuyang]

Hi, I am trying to use VizSuite to visualize objectnav in ithor (tutorial object_nav_ithor_ppo_one_object.py). Basically, what I am trying to do is to modify the task_info and other memory variables same as in Robothor. Everything is almost done. Just one question, in class IThorEnvironment, the agent state is acquired by the method called get_agent_location(). And in class RoboThorEnvironment, it's called agent_state(). The format is below:
IThorEnvironment.get_agent_location() ->

{'x': 1.25, 'y': 0.900999128818512, 'z': -0.25, 'rotation': 270.0, 'horizon': 30.000003814697266, 'standing': True}

RoboThorEnvironment.agent_state() ->

{'x': 4.5, 'y': 0.9009997248649597, 'z': -2.75, 'rotation': {'x': -0.0, 'y': 180.0, 'z': 0.0}, 'horizon': 0.0}

I wonder how to change iThor's format into RoboThor's. And is there an easy way to visualize this task?

Thank you!

BTW, there is a bug in this line https://github.com/allenai/allenact/blob/main/allenact/utils/model_utils.py#L175. Variable use_agents is not declared before. Probably, should be True when entering if statement, False when entering else statement.

[Lucaweihs]

Hi @zhangfuyang ,

@jordis-ai2 has a better grasp of the visualization code than I do so I'll let him expand on the

And is there an easy way to visualize this task?
question, but I can address the agent state:

Both RoboTHOR and iTHOR are built on top of the AI2-THOR simulator and thus share the same underlying agent state (if you look at the implementations of those two methods you'll see that they are both just extracting information from AI2-THOR's metadata). The only things to keep in mind are:

1. You can convert from that RoboTHOR rotation to the iTHOR rotation simply by as RoboThorEnvironment.agent_state()["rotation"]["y"]. The "x" and "z" components of the rotation will actually always be 0 (they would correspond to the agent tilting to the side which isn't possible currently).
2. For the position information (i.e. IThorEnvironment.get_agent_location()[w] and RoboThorEnvironment.agent_state()[w] for w in ["x", "y", "z"]), everything is identical for both environments (note that the "x" and "z" coordinates correspond to the ground plane, "y" is up).
3. In the iTHOR environment the agent can be standing or crouching, this is indicated by the "standing" field. Agents in the RoboTHOR environment cannot stand or crouch so this field cannot be converted.

Also, perhaps important to note, there are two main "agent" types used in AI2-THOR: "locobot" and "default". The "locobot" agent is the one typically associated with RoboTHOR (and "default" with iTHOR) but actually both agents can be used interchangeably. In AllenAct RoboThorEnvironment assumes you want to use the "locobot" (and IThorEnvironment assumes you want to use "default") as that's what is typically done.

BTW, there is a bug in this line https://github.com/allenai/allenact/blob/main/allenact/utils/model_utils.py#L175. Variable use_agents is not declared before. Probably, should be True when entering if statement, False when entering else statement.
Great catch, I've fixed this now.

[jordis-ai2]

Hi @zhangfuyang,

Adding specialized trajectory visualization is still pending for iTHOR. Unless I'm mistaken, it should be possible to add AgentViewViz and plain TrajectoryViz out-of-the-box, but I will add a specialized trajectory visualization and double check there are no hiccups for the other visualization types.

[zhangfuyang]

Thanks, @Lucaweihs and @jordis-ai2. I finally make it works. I make some modifications to match the current visualization tools. I attach my files here, in case some other people also look into it. (I am using this version of the code https://github.com/allenai/allenact/tree/c120d18e4bbe223a2b06a63853d8536b7b9d68c2, things may be different in the future :))
my_ithor_tasks.py.txt
my_ithor_task_samplers.py.txt
object_nav_ithor_ppo_one_object.py.txt

Just remove the ".txt" in the end after download (seems github doesn't allow upload python script).

[jordis-ai2]

Thanks, @zhangfuyang!

I actually created a PR with some fixes for visualization with iTHOR (including a specialized trajectory viewer). Let us know if it works for you if you have interest (it's still under review and as such subject to improvements).