Getting it to work on Linux
Closed this issue · 4 comments
iboates commented
Hi
I have installed the dependencies (including Blender) but I am not able to run the generation script because setup.txt is not found. There is a comment hinting that it should be in a Windows-based Anaconda env directory. I am also using Anaconda, but on Ubuntu Linux. I tried looking in the environment folder which I created for this project, but I can't find a setup.txt anywhere.
But because the path is relative I would assume that it should be found in the repository root. So maybe it is just missing?
Any help is appreciated.
Full stacktrace:
(building-dataset-generator) isaek@demoman1:~/PycharmProjects/Building-Dataset-Generator$ blender setup.blend --python dataset.py
Read prefs: /home/isaek/.config/blender/2.92/config/userpref.blend
Read blend: /home/isaek/PycharmProjects/Building-Dataset-Generator/setup.blend
Traceback (most recent call last):
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/dataset.py", line 13, in <module>
from annotation import Annotation
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/annotation.py", line 10, in <module>
from dataset_config import *
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/dataset_config.py", line 54, in <module>
SCRIPT_PATH = open('setup.txt').read()[:-1] # '"D:\ProgramFiles\Anaconda\envs\py37\Lib\site-packages"
FileNotFoundError: [Errno 2] No such file or directory: 'setup.txt'
iboates commented
This is a long update with a lot of errors so I'll just start with the headline:
It works! 🎉
Ok so I realized that this setup.py file should just point to the site-packages folder for the target environment.
My first environment would not work at all, throwing some esoteric low-level pandas errors. I was using Python 3.8. But your environment was named py37 so using that as a hint I re-initlaized a new conda environment with Python 3.7. That led me to this error:
(building-dataset-generator2) isaek@demoman1:~/PycharmProjects/Building-Dataset-Generator$ blender setup.blend --python dataset.py
Read prefs: /home/isaek/.config/blender/2.92/config/userpref.blend
Read blend: /home/isaek/PycharmProjects/Building-Dataset-Generator/setup.blend
<bpy_struct, Object("Camera") at 0x7f5366aaa208>
<bpy_struct, Object("Light") at 0x7f5366aaa808>
Traceback (most recent call last):
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/dataset.py", line 98, in <module>
d.populate()
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/dataset.py", line 49, in populate
lightmanager = LightManager()
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/light.py", line 10, in __init__
self.object = bpy.data.objects[self.name]
KeyError: 'bpy_prop_collection[key]: key "Sun" not found'
Saved session recovery to '/tmp/quit.blend'
Clearly it is complaining about a key not existing in the blender Python library. My guess is that it has to do with a version conflict. I printed the contents of bpy.data.objects and realized that it was the contents of the blender scene. My light was not named Sun, it was just named Light. So I changed that and got this error:
(building-dataset-generator2) isaek@demoman1:~/PycharmProjects/Building-Dataset-Generator$ blender setup.blend --python dataset.py
Read prefs: /home/isaek/.config/blender/2.92/config/userpref.blend
Read blend: /home/isaek/PycharmProjects/Building-Dataset-Generator/setup.blend
<bpy_struct, Object("Camera") at 0x7f5a8eaaa208>
<bpy_struct, Object("Light") at 0x7f5a8eaaa808>
Traceback (most recent call last):
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/dataset.py", line 98, in <module>
d.populate()
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/dataset.py", line 52, in populate
lightmanager.make()
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/light.py", line 15, in make
self._make()
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/light.py", line 18, in _make
self.light.angle = np.radians(np.random.randint(0, 75) + np.random.random())
AttributeError: 'PointLight' object has no attribute 'angle'
Saved session recovery to '/tmp/quit.blend'
I didn't really know what to do about this so out of desperation I just replaced the entire contents of LightManager._make() with pass. And it actually (mostly) worked, except it didn't generate pointclouds (probably just due to a missing folder), and it also wasn't able to find the texture files (which looks like some typical Unix/Windows path mangling):
(building-dataset-generator2) isaek@demoman1:~/PycharmProjects/Building-Dataset-Generator$ blender setup.blend --python dataset.py
Read prefs: /home/isaek/.config/blender/2.92/config/userpref.blend
Read blend: /home/isaek/PycharmProjects/Building-Dataset-Generator/setup.blend
<bpy_struct, Object("Camera") at 0x7fb182cdc208>
<bpy_struct, Object("Light") at 0x7fb182cdc808>
Failed to load Diffuse texture of Brick
ValueError("'//Textures\\\\Brick\\\\Diffuse.png' is not in list")
Failed to load Normal texture of Brick
ValueError("'//Textures\\\\Brick\\\\Normal.png' is not in list")
Failed to load Displacement texture of Brick
ValueError("'//Textures\\\\Brick\\\\Displacement.png' is not in list")
window | Glass
window | Glass
( 0.0000 sec | 0.0000 sec) Importing OBJ 'Modules/balcony/balcony1.obj'...
( 0.0001 sec | 0.0001 sec) Parsing OBJ file...
( 0.0057 sec | 0.0056 sec) Done, loading materials and images...
( 0.0074 sec | 0.0072 sec) Done, building geometries (verts:441 faces:195 materials: 1 smoothgroups:1) ...
( 0.0153 sec | 0.0151 sec) Done.
( 0.0153 sec | 0.0153 sec) Finished importing: 'Modules/balcony/balcony1.obj'
Progress: 100.00%
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Failed to load Diffuse texture of Metal_panels
ValueError("'//Textures\\\\Metal_panels\\\\Diffuse.png' is not in list")
Failed to load Normal texture of Metal_panels
ValueError("'//Textures\\\\Metal_panels\\\\Normal.png' is not in list")
Failed to load Displacement texture of Metal_panels
ValueError("'//Textures\\\\Metal_panels\\\\Displacement.png' is not in list")
balcony | Metal_panels
Info: Deleted 1 object(s)
Info: Deleted 1 object(s)
Info: Deleted 1 object(s)
Info: Deleted 1 object(s)
Info: Deleted 1 object(s)
Info: Deleted 1 object(s)
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Failed to load Diffuse texture of Metal_panels
ValueError("'//Textures\\\\Metal_panels\\\\Diffuse.png' is not in list")
Failed to load Normal texture of Metal_panels
ValueError("'//Textures\\\\Metal_panels\\\\Normal.png' is not in list")
Failed to load Displacement texture of Metal_panels
ValueError("'//Textures\\\\Metal_panels\\\\Displacement.png' is not in list")
balcony | Metal_panels
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Info: Applied modifier was not first, result may not be as expected
Info: Deleted 1 object(s)
Failed to load Diffuse texture of Brick3
ValueError("'//Textures\\\\Brick3\\\\Diffuse.png' is not in list")
Failed to load Normal texture of Brick3
ValueError("'//Textures\\\\Brick3\\\\Normal.png' is not in list")
Failed to load Displacement texture of Brick3
ValueError("'//Textures\\\\Brick3\\\\Displacement.png' is not in list")
balcony | Brick3
roof | Metall
[90.4991732195902, 0.0, 135.5519889095864]
( 0.0001 sec | 0.0000 sec) OBJ Export path: '/home/isaek/PycharmProjects/Building-Dataset-Generator/Models/0.obj'
( 0.0010 sec | 0.0007 sec) Finished writing geometry of 'volume'.
( 0.0016 sec | 0.0005 sec) Finished writing geometry of 'window.001'.
( 0.0021 sec | 0.0005 sec) Finished writing geometry of 'window.002'.
( 0.0027 sec | 0.0005 sec) Finished writing geometry of 'window.003'.
( 0.0033 sec | 0.0006 sec) Finished writing geometry of 'window.004'.
( 0.0039 sec | 0.0005 sec) Finished writing geometry of 'window.005'.
( 0.0044 sec | 0.0005 sec) Finished writing geometry of 'window.006'.
( 0.0050 sec | 0.0006 sec) Finished writing geometry of 'window.007'.
( 0.0057 sec | 0.0006 sec) Finished writing geometry of 'window.015'.
( 0.0063 sec | 0.0006 sec) Finished writing geometry of 'window.016'.
( 0.0069 sec | 0.0005 sec) Finished writing geometry of 'window.017'.
( 0.0074 sec | 0.0005 sec) Finished writing geometry of 'window.018'.
( 0.0080 sec | 0.0006 sec) Finished writing geometry of 'window.019'.
( 0.0086 sec | 0.0005 sec) Finished writing geometry of 'window.020'.
( 0.0092 sec | 0.0006 sec) Finished writing geometry of 'window.021'.
( 0.0097 sec | 0.0005 sec) Finished writing geometry of 'window.022'.
( 0.0103 sec | 0.0005 sec) Finished writing geometry of 'window.023'.
( 0.0108 sec | 0.0005 sec) Finished writing geometry of 'window.024'.
( 0.0113 sec | 0.0005 sec) Finished writing geometry of 'window.025'.
( 0.0119 sec | 0.0005 sec) Finished writing geometry of 'window.026'.
( 0.0124 sec | 0.0005 sec) Finished writing geometry of 'window.027'.
( 0.0130 sec | 0.0006 sec) Finished writing geometry of 'window.028'.
( 0.0135 sec | 0.0005 sec) Finished writing geometry of 'window.029'.
( 0.0141 sec | 0.0005 sec) Finished writing geometry of 'window.030'.
( 0.0146 sec | 0.0005 sec) Finished writing geometry of 'window.031'.
( 0.0152 sec | 0.0005 sec) Finished writing geometry of 'window.032'.
( 0.0157 sec | 0.0005 sec) Finished writing geometry of 'window.033'.
( 0.0162 sec | 0.0005 sec) Finished writing geometry of 'window.034'.
( 0.0168 sec | 0.0006 sec) Finished writing geometry of 'window.035'.
( 0.0174 sec | 0.0005 sec) Finished writing geometry of 'window.036'.
( 0.0179 sec | 0.0005 sec) Finished writing geometry of 'window.037'.
( 0.0185 sec | 0.0005 sec) Finished writing geometry of 'window.038'.
( 0.0190 sec | 0.0005 sec) Finished writing geometry of 'window.039'.
( 0.0195 sec | 0.0005 sec) Finished writing geometry of 'window.040'.
( 0.0201 sec | 0.0005 sec) Finished writing geometry of 'window.041'.
( 0.0207 sec | 0.0006 sec) Finished writing geometry of 'window.042'.
( 0.0212 sec | 0.0005 sec) Finished writing geometry of 'window.043'.
( 0.0217 sec | 0.0005 sec) Finished writing geometry of 'window.044'.
( 0.0223 sec | 0.0005 sec) Finished writing geometry of 'window.045'.
( 0.0228 sec | 0.0005 sec) Finished writing geometry of 'window.046'.
( 0.0234 sec | 0.0005 sec) Finished writing geometry of 'window.047'.
( 0.0239 sec | 0.0005 sec) Finished writing geometry of 'window.048'.
( 0.0245 sec | 0.0006 sec) Finished writing geometry of 'window.049'.
( 0.0538 sec | 0.0292 sec) Finished writing geometry of 'balcony1.001'.
( 0.0830 sec | 0.0292 sec) Finished writing geometry of 'balcony1.002'.
( 0.1120 sec | 0.0290 sec) Finished writing geometry of 'balcony1.003'.
( 0.1405 sec | 0.0285 sec) Finished writing geometry of 'balcony1.004'.
( 0.1696 sec | 0.0291 sec) Finished writing geometry of 'balcony1.005'.
( 0.1982 sec | 0.0286 sec) Finished writing geometry of 'balcony1.006'.
( 0.2282 sec | 0.0299 sec) Finished writing geometry of 'balcony1.007'.
( 0.2569 sec | 0.0287 sec) Finished writing geometry of 'balcony1.008'.
( 0.2858 sec | 0.0289 sec) Finished writing geometry of 'balcony1.010'.
( 0.3144 sec | 0.0286 sec) Finished writing geometry of 'balcony1.011'.
( 0.3434 sec | 0.0289 sec) Finished writing geometry of 'balcony1.012'.
( 0.3719 sec | 0.0286 sec) Finished writing geometry of 'balcony1.013'.
( 0.4008 sec | 0.0289 sec) Finished writing geometry of 'balcony1.014'.
( 0.4295 sec | 0.0286 sec) Finished writing geometry of 'balcony1.015'.
( 0.4898 sec | 0.0602 sec) Finished writing geometry of 'balcony1.016'.
( 0.5186 sec | 0.0288 sec) Finished writing geometry of 'balcony1.009'.
( 0.5474 sec | 0.0288 sec) Finished writing geometry of 'balcony1.018'.
( 0.5759 sec | 0.0284 sec) Finished writing geometry of 'balcony1.019'.
( 0.6047 sec | 0.0289 sec) Finished writing geometry of 'balcony1.020'.
( 0.6335 sec | 0.0287 sec) Finished writing geometry of 'balcony1.021'.
( 0.6623 sec | 0.0288 sec) Finished writing geometry of 'balcony1.022'.
( 0.6909 sec | 0.0285 sec) Finished writing geometry of 'balcony1.023'.
( 0.7198 sec | 0.0289 sec) Finished writing geometry of 'balcony1.024'.
( 0.7486 sec | 0.0287 sec) Finished writing geometry of 'balcony1.025'.
( 0.7776 sec | 0.0289 sec) Finished writing geometry of 'balcony1.026'.
( 0.8061 sec | 0.0285 sec) Finished writing geometry of 'balcony1.027'.
( 0.8351 sec | 0.0290 sec) Finished writing geometry of 'balcony1.028'.
( 0.8638 sec | 0.0286 sec) Finished writing geometry of 'balcony1.029'.
( 0.8927 sec | 0.0289 sec) Finished writing geometry of 'balcony1.030'.
( 0.9212 sec | 0.0285 sec) Finished writing geometry of 'balcony1.031'.
( 0.9502 sec | 0.0289 sec) Finished writing geometry of 'balcony1.032'.
( 0.9786 sec | 0.0284 sec) Finished writing geometry of 'balcony1.033'.
( 1.0078 sec | 0.0291 sec) Finished writing geometry of 'balcony1.034'.
( 1.0363 sec | 0.0285 sec) Finished writing geometry of 'balcony1.035'.
( 1.0653 sec | 0.0290 sec) Finished writing geometry of 'balcony1.036'.
( 1.0939 sec | 0.0285 sec) Finished writing geometry of 'balcony1.037'.
( 1.1546 sec | 0.0607 sec) Finished writing geometry of 'balcony1.038'.
( 1.1836 sec | 0.0289 sec) Finished writing geometry of 'balcony1.039'.
( 1.2127 sec | 0.0291 sec) Finished writing geometry of 'balcony1.040'.
( 1.2424 sec | 0.0297 sec) Finished writing geometry of 'balcony1.041'.
( 1.2433 sec | 0.0008 sec) Finished writing geometry of 'roof'.
( 1.2433 sec | 0.0000 sec) Finished writing geometry of 'Light'.
( 1.2434 sec | 0.0000 sec) Finished writing geometry of 'Camera'.
( 1.2434 sec | 0.0000 sec) Finished writing geometry of 'Camera.001'.
( 1.2435 sec | 1.2434 sec) Finished exporting geometry, now exporting materials
NO NODES! 100.00%
( 1.2445 sec | 1.2443 sec) OBJ Export Finished
Progress: 100.00%
Python: Traceback (most recent call last):
File "/snap/blender/111/2.92/scripts/addons/io_mesh_ply/__init__.py", line 174, in execute
export_ply.save(context, **keywords)
File "/snap/blender/111/2.92/scripts/addons/io_mesh_ply/export_ply.py", line 263, in save
use_colors,
File "/snap/blender/111/2.92/scripts/addons/io_mesh_ply/export_ply.py", line 155, in save_mesh
with open(filepath, "wb") as file:
FileNotFoundError: [Errno 2] No such file or directory: '/home/isaek/PycharmProjects/Building-Dataset-Generator/PointCloud/0.ply'
location: /snap/blender/111/2.92/scripts/modules/bpy/ops.py:132
Error: Python: Traceback (most recent call last):
File "/snap/blender/111/2.92/scripts/addons/io_mesh_ply/__init__.py", line 174, in execute
export_ply.save(context, **keywords)
File "/snap/blender/111/2.92/scripts/addons/io_mesh_ply/export_ply.py", line 263, in save
use_colors,
File "/snap/blender/111/2.92/scripts/addons/io_mesh_ply/export_ply.py", line 155, in save_mesh
with open(filepath, "wb") as file:
FileNotFoundError: [Errno 2] No such file or directory: '/home/isaek/PycharmProjects/Building-Dataset-Generator/PointCloud/0.ply'
location: /snap/blender/111/2.92/scripts/modules/bpy/ops.py:132
Traceback (most recent call last):
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/dataset.py", line 98, in <module>
d.populate()
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/dataset.py", line 82, in populate
building.save(i, ext='ply')
File "/home/isaek/PycharmProjects/Building-Dataset-Generator/generator.py", line 123, in save
use_selection=False)
File "/snap/blender/111/2.92/scripts/modules/bpy/ops.py", line 132, in __call__
ret = _op_call(self.idname_py(), None, kw)
RuntimeError: Error: Python: Traceback (most recent call last):
File "/snap/blender/111/2.92/scripts/addons/io_mesh_ply/__init__.py", line 174, in execute
export_ply.save(context, **keywords)
File "/snap/blender/111/2.92/scripts/addons/io_mesh_ply/export_ply.py", line 263, in save
use_colors,
File "/snap/blender/111/2.92/scripts/addons/io_mesh_ply/export_ply.py", line 155, in save_mesh
with open(filepath, "wb") as file:
FileNotFoundError: [Errno 2] No such file or directory: '/home/isaek/PycharmProjects/Building-Dataset-Generator/PointCloud/0.ply'
location: /snap/blender/111/2.92/scripts/modules/bpy/ops.py:132
Could you tell me exactly which Blender verison you are using? I think that this script may be very version-sensitive.
I also forked the repo containing my hack-fixes to make it work with my system
My system & environment specs:
# OS
Distributor ID: Ubuntu
Description: Ubuntu 18.04.5 LTS
Release: 18.04
Codename: bionic
# Python
Python 3.7.10
# Blender
Blender 2.92.0
build date: 2021-02-25
build time: 09:31:37
build commit date: 2021-02-24
build commit time: 16:25
build hash: 02948a2cab44
build platform: Linux
build type: Release
build c flags: -Wall -Wcast-align -Werror=implicit-function-declaration -Werror=return-type -Werror=vla -Wstrict-prototypes -Wmissing-prototypes -Wno-char-subscripts -Wno-unknown-pragmas -Wpointer-arith -Wunused-parameter -Wwrite-strings -Wlogical-op -Wundef -Winit-self -Wmissing-include-dirs -Wno-div-by-zero -Wtype-limits -Wformat-signedness -Wrestrict -Wnonnull -Wabsolute-value -Wuninitialized -Wredundant-decls -Wshadow -Wno-error=unused-but-set-variable -Wimplicit-fallthrough=5 -fuse-ld=gold -std=gnu11 -msse -pipe -fPIC -funsigned-char -fno-strict-aliasing -ffp-contract=off -msse2 -D_GLIBCXX_USE_CXX11_ABI=0
build c++ flags: -Wredundant-decls -Wall -Wno-invalid-offsetof -Wno-sign-compare -Wlogical-op -Winit-self -Wmissing-include-dirs -Wno-div-by-zero -Wtype-limits -Werror=return-type -Wno-char-subscripts -Wno-unknown-pragmas -Wpointer-arith -Wunused-parameter -Wwrite-strings -Wundef -Wformat-signedness -Wrestrict -Wuninitialized -Wundef -Wmissing-declarations -Wimplicit-fallthrough=5 -fuse-ld=gold -std=c++17 -msse -pipe -fPIC -funsigned-char -fno-strict-aliasing -ffp-contract=off -msse2 -D_GLIBCXX_USE_CXX11_ABI=0
build link flags: -Wl,--version-script='/home/sources/buildbot-worker-linux_centos7/linux_292/blender.git/source/creator/blender.map'
build system: CMake
iboates commented
I was able to squash all the errors but I still cannot get it to show up in blender with textures. Is that normal? I have attached a sample .obj and .mtl file. The .mtl file is defined via absolute paths so you will have to modify it on your end.
STASYA00 commented
the .mtl files are not automatically assigned to the .obj files, for that we will add an option to save .blend in case you need it! As the idea was to render the images with the textures and store the mesh as geometry.
Are the images rendered correctly with the textures?
Regarding the Blender and Python version: I will add them in the specifications, you are right, Blender is VERY version-specific. We used Blender 2.91.0 and python 3.7.0
We will look into the Sun error and will provide automatic folder generation to avoid the PointCloud issue
STASYA00 commented
@iboates I have fixed the issues you've mentioned (Sun and folders creation), thank you! And I checked, now it should be working.
Unfortunately, as I wrote before, .mtl files are not associated with the .obj automatically, but I have added the option to store .blend that keeps all the textures associated with the components.