Below is a demo video of navigating to the target table
(highlighted with red mask) in apartment_0
using habitat-sim
simulator (Link).
In order to navigate to certain area, the agent needs to have knowledge about the environment first.
Walking through the first floor (left figure) and second floor (right figure) in apartment_0
meanwhile collecting rgbd
and semantic
images at each steps.
Unproject the depth image into 3D point clouds, then align point clouds to the first frame using ICP algorithm
, we obtain the 3D rgb maps
of first floor (left figure) and second floor (right figure) in apartment_0
.
After creating maps for the environment, the agent can compute a collision-free trajectory and navigate to the target.
With 3D semantic map
(left figure), the agent knows about the location of each objects. In this example, we choose to navigate to the target table
(middle figure).
Using RRT algorithm
, the agent can compute a collision-free trajectory to the target starting from current position (right figure).
The agent will perfrom left
, right
, forward
these three actions according to the computed trajectory. The result can be seen on the top demo video.