Rendering script fails on Mac

[ThibaultGROUEIX]

Hi,

Thanks for your great work and for sharing all the code to generate the data.

I am trying to get the rotation matrix from the metadata. I've been struggling a bit with this so I tried reusing the rendering script you provide i.e generate.py to render a shapenet object with simple parameters. The script gives several errors on Mac, hence my question :

• did you render shapenet on Linux or Mac?

These errors are for instance :

  File "utils/render_utils.py", line 66, in make_area_lamp
    nodes = lamp.node_tree.nodes
AttributeError: 'NoneType' object has no attribute 'nodes'

This can be solved by setting :
lamp.use_nodes = True but there are other errors after this.

If you have any pointers to go from the metadata files to the rotation matrix, that would be great. I tried inversing the sequence of Euler-angle transformations in generate.py but the resulting matrix is not consistent with the renderings so far.
Cheers,
Thibault

[sstojanov]

Hi Thibault,

Thank you for your kind words and interest in our work! I have a few questions:

1. Are you trying to use our scripts with Blender 2.79b specifically? If so, then the scripts should work identically on Mac and on Linux. (more detail: the rendering itself was done on Linux, but we've done some developing/debugging on Mac and had the code work in both cases)
2. When you say rotation matrix, do you mean the the appropriate R, t (and K for projection) that would allow you, for example, to take a surface point cloud from the 3D model being rendered, transform and project it into the image? If this is the case, we have some code internally that builds on this release which allows for that functionality (dumps the camera intrinsic/extrinsic matrices in addition to the images/depths/normals for each frame). Do you think that would be useful for you?

Best,
Stefan

[ThibaultGROUEIX]

Hi @sstojanov,

Thanks for your fast reply.

1. Yes i was trying to use your script on Mac with a freshly installed Blender 2.79b from the link you provide. I am not sure why it fails then, but this was mostly done as a way to debug point 2.
2. Yes exactly, i am trying to write code that takes as input hard_vc_metadata.txt and metadata.txt and outputs the correct rotation matrix applied to the base shapenet model in a right-handed coordinate system (z forward, Y up, x left). I think your code could be very useful.

Thanks again!

[sstojanov]

Unfortunately I'm not sure why there are issues with the MacOS. For me it worked with 10.14.06 Mojave and previous versions. What version of MacOS are you using?

Regarding extracting the camera information, following is more detail. We are working on integrating this into the release, but the current answer is in terms of github gists.

Here are links to gists for an updated generate.py (link) and functions that need to be added to render_utils.py (link) for the camera rotation and calibration matrices to be output.

You can test these functions by doing the following:

1. Take an .obj from ShapeNet
2. Extract a point cloud using trimesh and save it as pointcloud3.npz
3. Convert that point cloud into the same orientation as the mesh in Blender using the following code.
4. Project the point cloud points using the following code

Following are some files that can help unit test/determine whether things are working properly

1. The following contains an example output rendering with the camera information output as cam.npz
   02691156.zip
2. Point cloud in original pointcloud3.npz and pointcloud3_R.npz Blender rendering orientation (this is before and after step 3 in the steps above) pointclouds.zip

If everything works correctly, the projected point cloud should correspond to the object in the image like in the example below. The image on the right was obtained following the testing steps 1 through 4 above.

Please let me know if you have further questions!
Best,
Stefan

[ThibaultGROUEIX]

Hi Stefan,
Loads of thanks for the detailed answer. I will go over the script and close the issue as soon as I reach the objective if that’s ok.
Best
Thibault

[sstojanov]

Hi Thibault, have you been able to make any progress on this?

Best,
Stefan

[marissachanlatte]

Hi Stefan,

I've been working with Thibault on this (he's on vacation) and yes, your scripts were very helpful. We were able to get what we needed.

Thank you!

Marissa

[sstojanov]

Glad to hear it! I'll go ahead and close the issue.