Trajectory computing not correct

[naivereseacher]

Hi Mike, I got another problem when running the example 1. As shown in the figure, the computed trajectory is outside the house.

The rendering is also wired. The first image is good, the same as you showed in the example. However, the rest of images are all black, which also makes sense if the trajectory is outside the house.

I wonder what causes this problem. Is the scene normalized in the code?

[mikeroberts3000]

Hi, you will need to provide some more detail so we can help effectively.

Where does the visualized camera trajectory come from, did you compute it yourself or did you obtain it from our released metadata? How exactly are you visualizing the camera trajectory and mesh, are you using our visualization code or did you write your own? How exactly did you obtain the triangle mesh? How exactly are you rendering those black images?

I'm closing the issue now but I'll re-open it if you can provide some additional detail.

[naivereseacher]

Hi Mike, thanks for your reply. Here is some information.

1. Where does the visualized camera trajectory come from, did you compute it yourself or did you obtain it from our released metadata?
   I compute this information myself using the released codes. I run the codes step by step, but the trajectory is not correct.

2. How exactly are you visualizing the camera trajectory and mesh, are you using our visualization code or did you write your own?
   I use the released visualization codes.
   python ../../code/python/tools/visualize_octomap.py --mesh_dir scenes/ai_001_001/_detail/mesh --octomap_dir scenes/ai_001_001/_detail/octomap --tmp_dir scenes/ai_001_001/_tmp

3. How exactly did you obtain the triangle mesh?
   I run the export scripts here to obtain the mesh. The difference is that I don't have 3ds max 2019 and the MaxPlus package is deprecated since then. Therefore, I use pymxs to run the Max scripts. I replace

def _eval(string, silent=False):
    if not silent:
        _print("[HYPERSIM: _DATASET_EXPORT_SCENES] Executing MAXScript: " + string)
    return MaxPlus.Core.EvalMAXScript(string)

with

def _eval(string, silent=False):
    if not silent:
        _print("[HYPERSIM: _DATASET_EXPORT_SCENES] Executing MAXScript: " + string)
    return pymxs.runtime.execute(string)

Another modification is that I have to change the argument 'FaceType=0' to 'FaceType=1' in obj export configuration, so that it can export triangular instead of quads.

4. How exactly are you rendering those black images?
   I run the following codes to render images
   python ../../code/python/tools/dataset_render_scene.py --dataset_dir . --scene_name ai_001_001 --camera_name cam_00 --render_pass geometry --save_image --frames 0:18

Here is some observation I have:

1. The preview image in '01_marketplace_dataset/scenes/ai_001_001/_detail/cam_00/preview' is not matched wit the rendered images.
   The first preview image is
   
   But the rendered image is
   

The second preview image is

The rendered image is

I think the coordinate of octomap (or object file) may not be the same with max file?

[mikeroberts3000]

I suspect that your OBJ has been exported incorrectly. In most 3D content creation tools, there is an option to swap mesh coordinates to maintain compatibility with other tools, so it is possible that one of these options is being used without realizing it. For our code to work correctly, we assume that the order and scale of all coordinates have been preserved in the OBJ file. Please try the following experiments and report your findings here.

1. Have you verified that your exported OBJ achieves numerical parity with what you see when you open the scene natively? For example, if an object is located at (x,y,z) == (100, 200, 300) when you open the scene natively, is that same object located at (x,y,z) == (100, 200, 300) in the OBJ file? Try doing a test where you export a simple scene with a few cubes and spheres at known locations, open the exported scene in a trusted viewer like MeshLab, and make sure the spheres and cubes are at the expected locations. Inspect the exported mesh visually and numerically.

2. If you visualize our publicly released camera trajectories (rather than the ones you re-computed yourself) using our visualization code, are our camera trajectories aligned with your exported mesh, or are our camera trajectories floating in empty space? If your mesh is exported correctly, then our camera trajectories will be aligned with it.

3. If you render images using our publicly released camera trajectories (rather than the ones you re-computed yourself), do you obtain images that match our publicly released images?

[naivereseacher]

Thanks Mike. I found the reason is I have to set 'FlipZyAxis=1' in 3ds Max 2021 (may also apply to later version) so that the coordinate system is unchanged. Otherwise, 3ds max will flip X, Y axis.

[mikeroberts3000]

Ah ha! I'm happy you figured it out. Can you please post your complete working gw_objexp.ini here in case someone else comes across this issue in future? It seems like the meaning of some of those config parameters have changed in the OBJ exporter since I originally released that exporting code in my personal fork. For example, in my fork, I set FaceType=0 to ensure that the scene gets exported into triangles, but you needed to set FaceType=1. I also set FlipZyAxis=0 to ensure that coordinates are numerically preserved, but you needed to set FlipZyAxis=1. Were those the only changes?

[naivereseacher]

I only modify these two variables. Here is my config.

[General]
UseLogging=1
Preset=<NONE>
MapSearchPath=

[Geometry]
FlipZyAxis=1
Shapes=0
ExportHiddenObjects=0
FaceType=1
TextureCoords=1
Normals=1
SmoothingGroups=0
ObjScale=1.000000

[Material]
UseMaterial=1
CreateMatLibrary=1
ForceBlackAmbient=0
UseMapPath=0
MapPath=./maps/
AutoMapChannel=0
MapChannel=1
ExtendedParams=0
ConvertBitmaps=0
RenderProcedural=1
ResizeMaps=0
ResizeMaps2n=0
MapFormat=3
MapSizeX=512
MapSizeY=512

[Output]
RelativeIndex=0
Target=0
Precision=20

[Optimize]
optVertex=1
optNormals=1
optTextureCoords=1