Not able to run blender_script.py and also how do I train on objaverse?

Question

Not able to run blender_script.py and also how do I train on objaverse?

Opened this issue 8 months ago · 24 comments

I'm not able to install the following two libraries mathutils and bpy:
For bpy it says:
ERROR: No matching distribution found for bpy
For mathutils it says:
ERROR: Failed building wheels for mathutils and
error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
My python version is 3.8.10

I have the objaverse dataset downloaded with objects, my question is do I have to run blender_script.py file for each .glb object in the dataset ?
Can you please provide the instructions on how to prepare the data properly for training.

Answer 1 · 2024-04-02T11:17:21.000Z

Hi,
Plz follow the data preparation instructions here.
You may need to use other scripts and follow the instructions here for distributed rendering https://github.com/allenai/objaverse-rendering. mathutils and bpy are built-in packages under blender.

Answer 2 · 2024-04-02T11:20:58.000Z

Just to clarify the blender_script.py works only for 1 object right?
Can you please also tell the format of the json file that I have to mention in train sample config file.

Answer 3 · 2024-04-02T11:21:15.000Z

Hi,
Plz follow the data preparation instructions here.
You may need to use other scripts and follow the instructions here for distributed rendering https://github.com/allenai/objaverse-rendering. mathutils and bpy are built-in packages under blender.

Thankyou

Answer 4 · 2024-04-02T11:27:09.000Z

Just to clarify the blender_script.py works only for 1 object right? Can you please also tell the format of the json file that I have to mention in train sample config file.

Yes, blender_script.py is for 1 object only.
The json file just contains a list of object ids. E.g. ["xxx", "yyy"]
You may refer to this comment #26 (comment) for the data structure.

Answer 5 · 2024-04-02T11:32:36.000Z

Just to clarify the blender_script.py works only for 1 object right? Can you please also tell the format of the json file that I have to mention in train sample config file.

Yes, blender_script.py is for 1 object only.
The json file just contains a list of object ids. E.g. ["xxx", "yyy"]
You may refer to this comment #26 (comment) for the data structure.

Thanks a lot 😄

Answer 6 · 2024-04-03T05:57:30.000Z

@ZexinHe whenever I try to train I'm getting the following error while on [TRAIN STEP] at 100th iteration, I tried to change the values of accum_steps and epochs in yaml file but at every 100th iteration the error occurs.
File "/home/Kunal/OpenLRM/openlrm/runners/train/lrm.py", line 418, in log_image_monitor
self.log_images({
Type error: Trainer.log_images() missing 1 required positional argument: 'log_kwargs'

Also what should I put in distributed type in accelerate-train.yaml if I'm a using single gpu.

I'm using one nvidia A6000.

Answer 7 · 2024-04-03T06:05:15.000Z

Hi,

It looks like some usage problem with self.log_images method. Could you plz provide more details about your environment? I don't see it a problem in mine.

Answer 8 · 2024-04-03T06:15:28.000Z

Hi,

It looks like some usage problem with self.log_images method. Could you plz provide more details about your environment? I don't see it a problem in mine.

Ubuntu 22
Python 3.10
Using python virtual environment

Answer 9 · 2024-04-03T06:30:54.000Z

One possible workaround here is to modify https://github.com/3DTopia/OpenLRM/blob/main/openlrm/runners/train/lrm.py#L418-L422.

self.log_images({ f'Images_split{split}/rendered': renders.unsqueeze(0), f'Images_split{split}/gt': gts.unsqueeze(0), f'Images_merged{split}': merged.unsqueeze(0), }, log_progress, {})

Just add an explicit empty {} to self.log_images method.

Answer 10 · 2024-04-03T09:04:29.000Z

One possible workaround here is to modify https://github.com/3DTopia/OpenLRM/blob/main/openlrm/runners/train/lrm.py#L418-L422.

self.log_images({ f'Images_split{split}/rendered': renders.unsqueeze(0), f'Images_split{split}/gt': gts.unsqueeze(0), f'Images_merged{split}': merged.unsqueeze(0), }, log_progress, {})

Just add an explicit empty {} to self.log_images method.

It works now, thanks

Answer 11 · 2024-04-03T09:56:29.000Z

Hey @ZexinHe I'm not able to run the convert_hf.ly file, I'm getting the following error:

omegaconf.errors.ConfigAttributeError: Missing key convert
full key: convert
object_type=dict

My command: python scripts/convert_hf.py --config "/home/Kunal/OpenLRM/configs/train-sample.yaml"

Also can I download the objaverse renderings used in training from anywhere, as it takes a lot of time for me to produce them through objaverse.
And also on how many objects the small model on hugging face (just objaverse) is trained on and how long did it take to train and on which and how many gpus?

Answer 12 · 2024-04-05T08:12:28.000Z

Hi,

I've updated some docs here https://github.com/3DTopia/OpenLRM/tree/main?tab=readme-ov-file#inference-on-trained-models.

Plz try to use python scripts/convert_hf.py --config <YOUR_EXACT_TRAINING_CONFIG> convert.global_step=null, which will convert the last training checkpoint by setting the convert.global_step to null.

Answer 13 · 2024-04-05T08:25:16.000Z

Hi,

I've updated some docs here https://github.com/3DTopia/OpenLRM/tree/main?tab=readme-ov-file#inference-on-trained-models.

Plz try to use python scripts/convert_hf.py --config <YOUR_EXACT_TRAINING_CONFIG> convert.global_step=null, which will convert the last training checkpoint by setting the convert.global_step to null.

Works now 😄

Answer 14 · 2024-04-16T05:06:19.000Z

Hi @kunalkathare , thank you for the post, it was very helpful.
By the way, since I'm very new to AI, don't know how to properly prepare data and run training.
I've prepared images through Objaverse Rendering, but not sure how to deal with the rest.
I've posted question on this issue.
Could you please check it once when you have time?
Thank you in advance.

Answer 15 · 2024-04-16T12:20:08.000Z

Hi @hayoung-jeremy, the blender_script.py renders pose, rgba, and intrinsic for each object, like this u have to run this script for each object, all this will be stored under views/ folder which will be the root dir in train sample yaml file, the json files should contain the list of uids of objects ["xxx","yyy",...].

This blender script file stores these 3 things under views/uid-of-object for your each object.

First install blender, then run this command for each object:
blender -b -P blender_script.py -- --object-path "your path to object"

Answer 16 · 2024-04-17T01:05:30.000Z

Thank you for the kind reply!
You mean I have to run the blender_script.py from this repository, not the one in the Objaverse Rendering repository, right?
I'll try it again, thank you so much!

Answer 17 · 2024-04-17T03:50:46.000Z

Thank you for the kind reply!
You mean I have to run the blender_script.py from this repository, not the one in the Objaverse Rendering repository, right?
I'll try it again, thank you so much!

Yes

Answer 18 · 2024-04-17T05:28:59.000Z

It works perfectly fine!

Thank you so much for your help @kunalkathare , I really appreciate it!
I needed to generate a dataset of 100 GLB files, so I created a process_glb_files.sh file that automatically runs a Blender script command for each file!
Share this with those who need it:

#!/bin/bash
DIRECTORY="./data" # replace it with your data folder path containing glb files

for glb_file in "$DIRECTORY"/*.glb; do
  echo "Processing $glb_file"
  blender -b -P scripts/data/objaverse/blender_script.py -- --object_path "$glb_file"
done

Answer 19 · 2024-04-17T09:00:17.000Z

Hi @kunalkathare , I was able to sucessfully run the training code.
However, I encountered an error after executing it and I'm not sure what I did wrong, so I am leaving a question here.
I generated a dataset of 100 pairs, each containing high-quality GLB files processed through OpenLRM's blender_script.py, and each pair includes rgba, pose, and intrinsics.npy information.
Afterwards, I ran the training code on Runpod’s multi GPU instance, which has four A100 SXM with 80GB VRAM.

I needed to set up the config file, but it was difficult to adjust because I don't understand what each item means, so I only specified the path and the number of GPUs before proceeding with the training as is.

Would it be meaningless to proceed with training using a dataset of 100 pairs?
When I tried with only 15 pairs, there wasn’t enough data, and it failed.
It would be helpful to know the minimum amount of data required and how to adjust the config file accordingly to match the amount of data.
Here is the link of the issue, I would appreciate it if you could check this when you have time.
Thank you.

Answer 20 · 2024-05-19T04:44:43.000Z

它工作得很好！非常感谢你的帮助 @kunalkathare，对此，我真的非常感激！我需要生成 100 个 GLB 文件的数据集，因此我创建了一个process_glb_files.sh文件，该文件会自动为每个文件运行 Blender 脚本命令！分享给有需要的人：
#!/bin/bash
DIRECTORY="./data" # replace it with your data folder path containing glb files

for glb_file in "$DIRECTORY"/*.glb; do
  echo "Processing $glb_file"
  blender -b -P scripts/data/objaverse/blender_script.py -- --object_path "$glb_file"
done

Hello! I also encountered this problem, my mathutils and bpy appeared red error, may I ask how you solved it? Your help will be greatly appreciated

Answer 21 · 2024-05-19T04:46:23.000Z

@hayoung-jeremy Hello! I also encountered this problem, my mathutils and bpy appeared red error, may I ask how you solved it? Your help will be greatly appreciated

Answer 22 · 2024-05-19T04:53:00.000Z

它工作得很好！非常感谢你的帮助 @kunalkathare，对此，我真的非常感激！我需要生成 100 个 GLB 文件的数据集，因此我创建了一个process_glb_files.sh文件，该文件会自动为每个文件运行 Blender 脚本命令！分享给有需要的人：
#!/bin/bash
DIRECTORY="./data" # replace it with your data folder path containing glb files

for glb_file in "$DIRECTORY"/*.glb; do
  echo "Processing $glb_file"
  blender -b -P scripts/data/objaverse/blender_script.py -- --object_path "$glb_file"
done
Hello! I also encountered this problem, my mathutils and bpy appeared red error, may I ask how you solved it? Your help will be greatly appreciated

Hi,
Plz follow the data preparation instructions here.
You may need to use other scripts and follow the instructions here for distributed rendering https://github.com/allenai/objaverse-rendering. mathutils and bpy are built-in packages under blender.

Answer 23 · 2024-05-19T05:00:02.000Z

它工作得很好！非常感谢你的帮助 @kunalkathare，对此，我真的非常感激！我需要生成 100 个 GLB 文件的数据集，因此我创建了一个process_glb_files.sh文件，该文件会自动为每个文件运行 Blender 脚本命令！分享给有需要的人：
#!/bin/bash
DIRECTORY="./data" # replace it with your data folder path containing glb files

for glb_file in "$DIRECTORY"/*.glb; do
  echo "Processing $glb_file"
  blender -b -P scripts/data/objaverse/blender_script.py -- --object_path "$glb_file"
done
Hello! I also encountered this problem, my mathutils and bpy appeared red error, may I ask how you solved it? Your help will be greatly appreciated
Hi, Plz follow the data preparation instructions here. You may need to use other scripts and follow the instructions here for distributed rendering https://github.com/allenai/objaverse-rendering. mathutils and bpy are built-in packages under blender.

Do you mean to use the blender_script from this URL instead of openlrm? I have a lot of knowledge is not very understanding at the beginning,Thank you for your help.

Answer 24 · 2024-05-19T05:01:16.000Z

它工作得很好！非常感谢你的帮助 @kunalkathare，对此，我真的非常感激！我需要生成 100 个 GLB 文件的数据集，因此我创建了一个process_glb_files.sh文件，该文件会自动为每个文件运行 Blender 脚本命令！分享给有需要的人：
#!/bin/bash
DIRECTORY="./data" # replace it with your data folder path containing glb files

for glb_file in "$DIRECTORY"/*.glb; do
  echo "Processing $glb_file"
  blender -b -P scripts/data/objaverse/blender_script.py -- --object_path "$glb_file"
done
Hello! I also encountered this problem, my mathutils and bpy appeared red error, may I ask how you solved it? Your help will be greatly appreciated
Hi, Plz follow the data preparation instructions here. You may need to use other scripts and follow the instructions here for distributed rendering https://github.com/allenai/objaverse-rendering. mathutils and bpy are built-in packages under blender.
Do you mean to use the blender_script from this URL instead of openlrm? I have a lot of knowledge is not very understanding at the beginning,Thank you for your help.

Use from OpenLRM