Is it possible to use HSSD for object navigation

[ConnerQiu]

Currently I see that in project/habitat_objectnav only includes hm3d v0.2. I wonder if it is possible to use HSSD for object nav as HSSD has more object categories(129 if I remember right), while hm3d only has 6 object categories and mp3d has 21 object categories.

I think it is possible, maybe we need to create some objectnav episodes that follow similar step as how to create ovmm episode. If possible, can you share some experience or codes about the episode creation.

Or more generally, can you give some guidances about how to use HSSD for object nav. Great thanks in advance~

[yvsriram]

Hi, there is an ObjectNav task defined on HSSD (https://github.com/3dlg-hcvc/hssd), but these episodes use only 6 object categories (@mukulkhanna can confirm this bit).

1. You can directly use the OVMM episodes to perform your ObjectNav experiments. The first sub-task in OVMM is navigating to objects (108 seen and 21 unseen categories) on top of given start receptacles. This differs from standard ObjectNav which doesn't have the additional constraint of reaching objects on top of receptacles. We have viewpoints in our episodes (candidate_objects field) only for the set of candidate objects that are on top of a given receptacle.

2. If you are interested in running ObjectNav experiments where you don't have the constraint of navigating to objects on top of receptacles (candidate_objects_hard will contain goals), you can follow this script and generate viewpoints for all candidate objects by
   using candidate_objects_hard instead of candidate_objects here:
   https://github.com/facebookresearch/habitat-lab/blob/45aa489a84a853cb10b9e7c87383262831a6bd22/habitat-lab/habitat/datasets/rearrange/modify_episodes_for_object_rearrange.py#L364

3. But if you decide to generate your own episodes from scratch, you can make modifications to the OVMM episode generation scripts here: https://github.com/facebookresearch/habitat-lab/tree/ovmm_main_merge/habitat-lab/habitat/datasets/rearrange#generating-episodes-for-open-vocab-mobile-manipulation-task

[ConnerQiu]

Wow, thanks!! These information are super helpful! I want to generate the object nav episode with expaned object category, so I think modifying the code of OVMM episode generation to object nav generation should works~