Fail to install FRNN
Closed this issue · 17 comments
better1593 commented
Hi Damien,
Thanks for the brand new v2!
I am trying to run install.sh to install all the dependencies, but there seems to be an error when it comes to FRNN installation.
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/utils/cpp_extension.py:425: UserWarning: There are no g++ version bounds defined for CUDA version 11.8
warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
and here is a part of the error info:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h: In instantiation of ‘void torch::nn::ConvNdImpl<D, Derived>::reset_parameters() [with long unsigned int D = 1; Derived = torch::nn::ConvTranspose1dImpl]’:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:101:5: required from ‘void torch::nn::ConvNdImpl<D, Derived>::reset() [with long unsigned int D = 1; Derived = torch::nn::ConvTranspose1dImpl]’
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:33:8: required from here
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:105:27: error: cannot convert ‘const torch::enumtype::kFanIn’ to ‘int’
105 | init::kaiming_uniform_(
| ~~~~~~~~~~~~~~~~~~~~~~^
| |
| const torch::enumtype::kFanIn
106 | weight,
| ~~~~~~~
107 | /*a=*/std::sqrt(5)); // NOLINT(cppcoreguidelines-avoid-magic-numbers)
| ~~~~~~~~~~~~~~~~~~~
In file included from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:5,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/all.h:16,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/extension.h:5,
from /root/autodl-tmp/superpoint_transformer/src/dependencies/FRNN/frnn/csrc/bruteforce/bruteforce_cpu.cpp:2:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/init.h:99:17: note: initializing argument 3 of ‘at::Tensor torch::nn::init::kaiming_uniform_(at::Tensor, double, int, int)’
99 | FanModeType mode = torch::kFanIn,
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
In file included from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules.h:20,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:7,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/all.h:16,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/extension.h:5,
from /root/autodl-tmp/superpoint_transformer/src/dependencies/FRNN/frnn/csrc/bruteforce/bruteforce_cpu.cpp:2:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h: In instantiation of ‘void torch::nn::ConvNdImpl<D, Derived>::reset_parameters() [with long unsigned int D = 3; Derived = torch::nn::Conv3dImpl]’:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:101:5: required from ‘void torch::nn::ConvNdImpl<D, Derived>::reset() [with long unsigned int D = 3; Derived = torch::nn::Conv3dImpl]’
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:33:8: required from here
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:105:27: error: cannot convert ‘const torch::enumtype::kFanIn’ to ‘int’
105 | init::kaiming_uniform_(
| ~~~~~~~~~~~~~~~~~~~~~~^
| |
| const torch::enumtype::kFanIn
106 | weight,
| ~~~~~~~
107 | /*a=*/std::sqrt(5)); // NOLINT(cppcoreguidelines-avoid-magic-numbers)
| ~~~~~~~~~~~~~~~~~~~
In file included from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:5,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/all.h:16,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/extension.h:5,
from /root/autodl-tmp/superpoint_transformer/src/dependencies/FRNN/frnn/csrc/bruteforce/bruteforce_cpu.cpp:2:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/init.h:99:17: note: initializing argument 3 of ‘at::Tensor torch::nn::init::kaiming_uniform_(at::Tensor, double, int, int)’
99 | FanModeType mode = torch::kFanIn,
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
In file included from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules.h:20,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:7,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/all.h:16,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/extension.h:5,
from /root/autodl-tmp/superpoint_transformer/src/dependencies/FRNN/frnn/csrc/bruteforce/bruteforce_cpu.cpp:2:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h: In instantiation of ‘void torch::nn::ConvNdImpl<D, Derived>::reset_parameters() [with long unsigned int D = 2; Derived = torch::nn::Conv2dImpl]’:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:101:5: required from ‘void torch::nn::ConvNdImpl<D, Derived>::reset() [with long unsigned int D = 2; Derived = torch::nn::Conv2dImpl]’
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:33:8: required from here
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:105:27: error: cannot convert ‘const torch::enumtype::kFanIn’ to ‘int’
105 | init::kaiming_uniform_(
| ~~~~~~~~~~~~~~~~~~~~~~^
| |
| const torch::enumtype::kFanIn
106 | weight,
| ~~~~~~~
107 | /*a=*/std::sqrt(5)); // NOLINT(cppcoreguidelines-avoid-magic-numbers)
| ~~~~~~~~~~~~~~~~~~~
In file included from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:5,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/all.h:16,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/extension.h:5,
from /root/autodl-tmp/superpoint_transformer/src/dependencies/FRNN/frnn/csrc/bruteforce/bruteforce_cpu.cpp:2:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/init.h:99:17: note: initializing argument 3 of ‘at::Tensor torch::nn::init::kaiming_uniform_(at::Tensor, double, int, int)’
99 | FanModeType mode = torch::kFanIn,
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
In file included from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules.h:20,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:7,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/all.h:16,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/extension.h:5,
from /root/autodl-tmp/superpoint_transformer/src/dependencies/FRNN/frnn/csrc/bruteforce/bruteforce_cpu.cpp:2:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h: In instantiation of ‘void torch::nn::ConvNdImpl<D, Derived>::reset_parameters() [with long unsigned int D = 1; Derived = torch::nn::Conv1dImpl]’:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:101:5: required from ‘void torch::nn::ConvNdImpl<D, Derived>::reset() [with long unsigned int D = 1; Derived = torch::nn::Conv1dImpl]’
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:33:8: required from here
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h:105:27: error: cannot convert ‘const torch::enumtype::kFanIn’ to ‘int’
105 | init::kaiming_uniform_(
| ~~~~~~~~~~~~~~~~~~~~~~^
| |
| const torch::enumtype::kFanIn
106 | weight,
| ~~~~~~~
107 | /*a=*/std::sqrt(5)); // NOLINT(cppcoreguidelines-avoid-magic-numbers)
| ~~~~~~~~~~~~~~~~~~~
In file included from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:5,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/all.h:16,
from /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/extension.h:5,
from /root/autodl-tmp/superpoint_transformer/src/dependencies/FRNN/frnn/csrc/bruteforce/bruteforce_cpu.cpp:2:
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/init.h:99:17: note: initializing argument 3 of ‘at::Tensor torch::nn::init::kaiming_uniform_(at::Tensor, double, int, int)’
99 | FanModeType mode = torch::kFanIn,
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1
better1593 commented
Hi Damien,
I tried to change torch version 2.3.0 into 2.0.0 and it worked.
But when I tried to run the command python src/train.py experiment=semantic/dales, I received a prompt saying I didn't insatll the torch_geometric module. So, I directly installed torch_geometric using pip install torch_geometric and attempted to run python src/train.py experiment=semantic/dales again. This time, I encountered an error as follows:
Processing...
/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/deprecation.py:26: UserWarning: 'makedirs' is deprecated, use 'os.makedirs(path, exist_ok=True)' instead
warnings.warn(out)
0%| | 0/261 [00:05<?, ?it/s]
[�[36m2024-03-02 12:56:24,511�[39m][�[34msrc.utils.utils�[39m][�[31mERROR�[39m] -
Traceback (most recent call last):
...
File "/root/autodl-tmp/superpoint_transformer/src/data/data.py", line 150, in v_edge_keys
return [k for k in self.keys if k.startswith('v_edge_')]
TypeError: 'method' object is not iterable
I suspect it's due to the version of torch_geomtric, so I rolled back to version 2.3.0 and the problem was resolved.
better1593 commented
Another error comes up😂. The preprocessing works fine, but when it starts training, I get this:
Traceback (most recent call last):
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1194, in run_node
return node.target(*args, **kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_ops.py", line 502, in __call__
return self._op(*args, **kwargs or {})
RuntimeError: Cannot call sizes() on tensor with symbolic sizes/strides
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1152, in get_fake_value
return wrap_fake_exception(
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 808, in wrap_fake_exception
return fn()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1153, in <lambda>
lambda: run_node(tx.output, node, args, kwargs, nnmodule)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1206, in run_node
raise RuntimeError(
RuntimeError: Failed running call_function torch_scatter.scatter_min(*(FakeTensor(FakeTensor(..., device='meta', size=(s0, s2)), cuda:0), FakeTensor(FakeTensor(..., device='meta', size=(s0,), dtype=torch.int64), cuda:0), 0, None, None), **{}):
Cannot call sizes() on tensor with symbolic sizes/strides
(scroll up for backtrace)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/autodl-tmp/superpoint_transformer/src/utils/utils.py", line 45, in wrap
metric_dict, object_dict = task_func(cfg=cfg)
File "src/train.py", line 114, in train
trainer.fit(model=model, datamodule=datamodule, ckpt_path=cfg.get("ckpt_path"))
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 543, in fit
call._call_and_handle_interrupt(
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 44, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 579, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 986, in _run
results = self._run_stage()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1030, in _run_stage
self._run_sanity_check()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1059, in _run_sanity_check
val_loop.run()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/loops/utilities.py", line 182, in _decorator
return loop_run(self, *args, **kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/loops/evaluation_loop.py", line 135, in run
self._evaluation_step(batch, batch_idx, dataloader_idx, dataloader_iter)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/loops/evaluation_loop.py", line 396, in _evaluation_step
output = call._call_strategy_hook(trainer, hook_name, *step_args)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 309, in _call_strategy_hook
output = fn(*args, **kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/strategies/strategy.py", line 412, in validation_step
return self.lightning_module.validation_step(*args, **kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 209, in _fn
return fn(*args, **kwargs)
File "/root/autodl-tmp/superpoint_transformer/src/models/semantic.py", line 519, in validation_step
loss, output = self.model_step(batch)
File "/root/autodl-tmp/superpoint_transformer/src/models/semantic.py", line 203, in model_step
output = self.step_single_run_inference(batch) \
File "/root/autodl-tmp/superpoint_transformer/src/models/semantic.py", line 300, in step_single_run_inference
output = self.forward(nag)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 209, in _fn
return fn(*args, **kwargs)
File "/root/autodl-tmp/superpoint_transformer/src/models/semantic.py", line 133, in forward
x = self.net(nag)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/root/autodl-tmp/superpoint_transformer/src/models/components/spt.py", line 451, in forward
x, diameter = self.first_stage(
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 337, in catch_errors
return callback(frame, cache_size, hooks)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 404, in _convert_frame
result = inner_convert(frame, cache_size, hooks)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 104, in _fn
return fn(*args, **kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 262, in _convert_frame_assert
return _compile(
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 163, in time_wrapper
r = func(*args, **kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 324, in _compile
out_code = transform_code_object(code, transform)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 445, in transform_code_object
transformations(instructions, code_options)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 311, in transform
tracer.run()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1726, in run
super().run()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 576, in run
and self.step()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 540, in step
getattr(self, inst.opname)(inst)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 342, in wrapper
return inner_fn(self, inst)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1014, in CALL_FUNCTION_KW
self.call_function(fn, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 474, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/variables/nn_module.py", line 244, in call_function
return tx.inline_user_function_return(
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1806, in inline_call
return cls.inline_call_(parent, func, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1862, in inline_call_
tracer.run()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 576, in run
and self.step()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 540, in step
getattr(self, inst.opname)(inst)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 342, in wrapper
return inner_fn(self, inst)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1014, in CALL_FUNCTION_KW
self.call_function(fn, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 474, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 291, in call_function
return super().call_function(tx, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 259, in call_function
return super().call_function(tx, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 92, in call_function
return tx.inline_user_function_return(
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1806, in inline_call
return cls.inline_call_(parent, func, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1862, in inline_call_
tracer.run()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 576, in run
and self.step()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 540, in step
getattr(self, inst.opname)(inst)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 342, in wrapper
return inner_fn(self, inst)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1014, in CALL_FUNCTION_KW
self.call_function(fn, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 474, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 259, in call_function
return super().call_function(tx, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 92, in call_function
return tx.inline_user_function_return(
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1806, in inline_call
return cls.inline_call_(parent, func, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1862, in inline_call_
tracer.run()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 576, in run
and self.step()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 540, in step
getattr(self, inst.opname)(inst)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 342, in wrapper
return inner_fn(self, inst)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 965, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 474, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 259, in call_function
return super().call_function(tx, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 92, in call_function
return tx.inline_user_function_return(
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1806, in inline_call
return cls.inline_call_(parent, func, args, kwargs)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1862, in inline_call_
tracer.run()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 576, in run
and self.step()
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 540, in step
getattr(self, inst.opname)(inst)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 342, in wrapper
return inner_fn(self, inst)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 965, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 474, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/variables/torch.py", line 548, in call_function
tensor_variable = wrap_fx_proxy(
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 754, in wrap_fx_proxy
return wrap_fx_proxy_cls(
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 789, in wrap_fx_proxy_cls
example_value = get_fake_value(proxy.node, tx)
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1173, in get_fake_value
raise TorchRuntimeError() from e
torch._dynamo.exc.TorchRuntimeError:
from user code:
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch_scatter/scatter.py", line 158, in scatter
return scatter_min(src, index, dim, out, dim_size)[0]
File "/root/miniconda3/envs/spt/lib/python3.8/site-packages/torch_scatter/scatter.py", line 65, in scatter_min
return torch.ops.torch_scatter.scatter_min(src, index, dim, out, dim_size)
Set torch._dynamo.config.verbose=True for more information
You can suppress this exception and fall back to eager by setting:
torch._dynamo.config.suppress_errors = True
The issue may still be related to the versions of torch and other dependencies😂.
drprojects commented
Hi @better1593 thanks for reporting these issues. I have not encountered these issues but they may be related to the CUDA or gcc version you use in the first place. We moved to more recent versions of torch and torch_geometric, in part for better support of the torch.compile() mechanism. Maybe FRNN lags behind in terms of support.
Before anything, does your system abide by the requirements specified in the README ? Can you please share your CUDA, gcc, and g++ versions ?
better1593 commented
Hi @better1593 thanks for reporting these issues. I have not encountered these issues but they may be related to the CUDA or gcc version you use in the first place. We moved to more recent versions of
torchandtorch_geometric, in part for better support of thetorch.compile()mechanism. Maybe FRNN lags behind in terms of support.Before anything, does your system abide by the requirements specified in the README ? Can you please share your CUDA, gcc, and g++ versions ?
Hi, I’m using CUDA version 11.8, gcc/g++ version 11.3.0.
drprojects commented
That's strange, I have a machine with the same specs and could install everything.
So right now, which versions do you have for the following libraries ?
- torch
- torch-geometric
- FRNN
Also, can you please try setting
compile: Falsein your train.yaml or eval.yaml config and see if that resolves the last error you encountered ?
noisyneighbour commented
I encountered the same issues as @better1593 when trying to set-up the environment using the v2 version of the code.
- Torch compilation failed due to this issue: To solve this in src/dependencies/FRNN I had to change extra_compile_args to:
extra_compile_args = {"cxx": ["-std=c++17"]} - torch_geometric was not installed, and "pip install torch_geometric" installed version 2.5.0, which caused the same error as shown above (TypeError: 'method' object is not iterable)
- Training using compile=True failed the dynamo error shown above. After setting compile=False, I am able to train normally.
drprojects commented
- Torch compilation failed due to open-mmlab/mmdeploy#2529: To solve this in src/dependencies/FRNN I had to change extra_compile_args to:
extra_compile_args = {"cxx": ["-std=c++17"]}
Can you please share where you set extra_compile_args = {"cxx": ["-std=c++17"]} so that I can look into improving the installation process ?
- torch_geometric was not installed, and "pip install torch_geometric" installed version 2.5.0, which caused the same error as shown above (TypeError: 'method' object is not iterable)
Which PyG version did you install in the end ? 2.3.0 ?
- Training using compile=True failed the dynamo error shown above. After setting compile=False, I am able to train normally.
Yes, I see that the dynamo does not seem to work well with torch_scatter. This was the reason why we did not use torch.compile so far but I understood this should have been solved in the latest versions of torch_geometric. Sadly, I do not have time to investigate the matter, so I will set compile: False by default for now. Any PR for making torch.compile() work with torch_scatter would be welcome 😊
noisyneighbour commented
- Torch compilation failed due to [Bug] #error C++17 or later compatible compiler is required to use PyTorch open-mmlab/mmdeploy#2529: To solve this in src/dependencies/FRNN I had to change extra_compile_args to:
extra_compile_args = {"cxx": ["-std=c++17"]}Can you please share where you set
extra_compile_args = {"cxx": ["-std=c++17"]}so that I can look into improving the installation process ?
Here is the line in FRNN setup.py
- torch_geometric was not installed, and "pip install torch_geometric" installed version 2.5.0, which caused the same error as shown above (TypeError: 'method' object is not iterable)
Which PyG version did you install in the end ? 2.3.0 ?
Yes, 2.3.0 was the version that worked for me. I used cuda 12.1 on an AWS EC2 G5 instance to build the environment.
better1593 commented
So right now, which versions do you have for the following libraries ?
Right now I‘m using:
- torch 2.0.0
- torch geometric 2.3.0
- FRNN 0.0.0
And as you and @noisyneighbour said, I can train normally after I setcompile=False.
drprojects commented
OK so to summarize:
- due to clashes between
torch.compileandtorch_scatter, we cannot usetorch.compilefor now 🥲 ➡️ I will setcompile: Falseby default - FRNN compilation error requires editing the FRNN
setup.pyscript ➡️ FRNN-dependent, please file an issue on the FRNN repo
Yarroudh commented
I had the same issue with custom dataset and setting compile to False resolved the issue. Thanks @drprojects for the great work.
drprojects commented
Thanks @Yarroudh for your feedback !
QingXia1994 commented
I still have the same problem when i run this project with 'compile': False.
Here is the Log:
├── datamodule
│ └── partition_hf:
│ - linearity
│ - planarity
│ - scattering
│ - elevation
│ point_hf:
│ - intensity
│ - linearity
│ - planarity
│ - scattering
│ - verticality
│ - elevation
│ segment_base_hf: []
│ segment_mean_hf: []
│ segment_std_hf: []
│ edge_hf:
│ - mean_off
│ - std_off
│ - mean_dist
│ - angle_source
│ - angle_target
│ - centroid_dir
│ - centroid_dist
│ - normal_angle
│ - log_length
│ - log_surface
│ - log_volume
│ - log_size
│ v_edge_hf: []
│ h_edge_hf_need_normal: true
│ v_edge_hf_need_normal: false
│ edge_hf_need_log_length: true
│ edge_hf_need_log_surface: true
│ edge_hf_need_log_volume: true
│ edge_hf_need_log_size: true
│ extra_point_hf: []
│ extra_segment_hf:
│ - normal
│ - log_length
│ - log_surface
│ - log_volume
│ - log_size
│ segment_hf: []
│ point_hf_preprocess:
│ - scattering
│ - planarity
│ - intensity
│ - verticality
│ - elevation
│ - linearity
│ segment_base_hf_preprocess:
│ - log_size
│ - log_volume
│ - normal
│ - log_surface
│ - log_length
│ segment_mean_hf_preprocess: []
│ segment_std_hf_preprocess: []
│ num_hf_point: 6
│ num_hf_segment: 0
│ num_hf_edge: 18
│ num_hf_v_edge: 0
│ obj_key:
│ - obj
│ point_full_res_key:
│ - sub
│ point_save_keys: null
│ point_no_save_keys:
│ - edge_index
│ - edge_attr
│ - neighbor_index
│ - neighbor_distance
│ - node_size
│ - grid_size
│ segment_save_keys: null
│ point_basic_load_keys:
│ - pos
│ - pos_offset
│ - 'y'
│ - obj
│ - super_index
│ segment_basic_load_keys:
│ - pos
│ - 'y'
│ - super_index
│ - sub
│ - edge_index
│ - edge_attr
│ point_load_keys:
│ - scattering
│ - obj
│ - planarity
│ - pos
│ - intensity
│ - 'y'
│ - verticality
│ - super_index
│ - elevation
│ - pos_offset
│ - linearity
│ segment_load_keys:
│ - edge_attr
│ - edge_index
│ - log_size
│ - pos
│ - log_volume
│ - sub
│ - 'y'
│ - normal
│ - super_index
│ - log_surface
│ - log_length
│ feat_size:
│ pos: 3
│ pos_room: 3
│ rgb: 3
│ hsv: 3
│ lab: 3
│ density: 1
│ linearity: 1
│ planarity: 1
│ scattering: 1
│ verticality: 1
│ normal: 3
│ length: 1
│ surface: 1
│ volume: 1
│ curvature: 1
│ elevation: 1
│ size: 1
│ intensity: 1
│ log_pos: 3
│ log_pos_room: 3
│ log_rgb: 3
│ log_hsv: 3
│ log_lab: 3
│ log_density: 1
│ log_linearity: 1
│ log_planarity: 1
│ log_scattering: 1
│ log_verticality: 1
│ log_normal: 3
│ log_length: 1
│ log_surface: 1
│ log_volume: 1
│ log_curvature: 1
│ log_elevation: 1
│ log_size: 1
│ mean_pos: 3
│ mean_pos_room: 3
│ mean_rgb: 3
│ mean_hsv: 3
│ mean_lab: 3
│ mean_density: 1
│ mean_linearity: 1
│ mean_planarity: 1
│ mean_scattering: 1
│ mean_verticality: 1
│ mean_normal: 3
│ mean_length: 1
│ mean_surface: 1
│ mean_volume: 1
│ mean_curvature: 1
│ mean_elevation: 1
│ mean_size: 1
│ mean_intensity: 1
│ std_pos: 3
│ std_pos_room: 3
│ std_rgb: 3
│ std_hsv: 3
│ std_lab: 3
│ std_density: 1
│ std_linearity: 1
│ std_planarity: 1
│ std_scattering: 1
│ std_verticality: 1
│ std_normal: 3
│ std_length: 1
│ std_surface: 1
│ std_volume: 1
│ std_curvature: 1
│ std_elevation: 1
│ std_size: 1
│ std_intensity: 1
│ mean_off: 3
│ std_off: 3
│ mean_dist: 1
│ angle_source: 1
│ angle_target: 1
│ centroid_dir: 3
│ centroid_dist: 1
│ normal_angle: 1
│ _target_: src.datamodules.dales.DALESDataModule
│ data_dir: superpoint_transformer/data/
│ num_classes: 8
│ stuff_classes:
│ - 0
│ - 1
│ instance: false
│ instance_k_max: 30
│ instance_radius: 0.1
│ min_instance_size: 100
│ mini: false
│ save_y_to_csr: true
│ save_pos_dtype: float32
│ save_fp_dtype: float16
│ in_memory: false
│ lite_preprocessing: true
│ load_full_res_idx: false
│ max_num_nodes: 50000
│ max_num_edges: 1000000
│ pre_transform:
│ - transform: SaveNodeIndex
│ params:
│ key: sub
│ - transform: DataTo
│ params:
│ device: cuda
│ - transform: GridSampling3D
│ params:
│ size: 0.1
│ hist_key: 'y'
│ hist_size: 9
│ - transform: KNN
│ params:
│ k: 25
│ r_max: 10
│ verbose: false
│ - transform: DataTo
│ params:
│ device: cpu
│ - transform: GroundElevation
│ params:
│ threshold: 5
│ scale: 20
│ - transform: PointFeatures
│ params:
│ keys:
│ - scattering
│ - planarity
│ - intensity
│ - verticality
│ - elevation
│ - linearity
│ k_min: 1
│ k_step: -1
│ k_min_search: 10
│ - transform: DataTo
│ params:
│ device: cuda
│ - transform: AdjacencyGraph
│ params:
│ k: 10
│ w: 1
│ - transform: ConnectIsolated
│ params:
│ k: 1
│ - transform: DataTo
│ params:
│ device: cpu
│ - transform: AddKeysTo
│ params:
│ keys:
│ - linearity
│ - planarity
│ - scattering
│ - elevation
│ to: x
│ delete_after: false
│ - transform: CutPursuitPartition
│ params:
│ regularization:
│ - 0.1
│ - 0.2
│ - 0.3
│ spatial_weight:
│ - 0.1
│ - 0.01
│ - 0.001
│ k_adjacency: 10
│ cutoff:
│ - 10
│ - 30
│ - 100
│ iterations: 15
│ parallel: true
│ verbose: false
│ - transform: NAGRemoveKeys
│ params:
│ level: all
│ keys: x
│ - transform: NAGTo
│ params:
│ device: cuda
│ - transform: SegmentFeatures
│ params:
│ n_min: 32
│ n_max: 128
│ keys:
│ - log_size
│ - log_volume
│ - normal
│ - log_surface
│ - log_length
│ mean_keys: []
│ std_keys: []
│ strict: false
│ - transform: RadiusHorizontalGraph
│ params:
│ k_min: 1
│ k_max: 30
│ gap:
│ - 5
│ - 30
│ - 30
│ se_ratio: 0.3
│ se_min: 20
│ cycles: 3
│ margin: 0.5
│ chunk_size:
│ - 1000000.0
│ - 100000.0
│ - 100000.0
│ halfspace_filter: true
│ bbox_filter: true
│ target_pc_flip: true
│ source_pc_sort: false
│ keys:
│ - mean_off
│ - std_off
│ - mean_dist
│ - transform: NAGTo
│ params:
│ device: cpu
│ train_transform: null
│ val_transform: null
│ test_transform: null
│ on_device_train_transform:
│ - transform: NodeSize
│ - transform: SampleSubNodes
│ params:
│ low: 0
│ high: 1
│ n_min: 32
│ n_max: 128
│ - transform: SampleRadiusSubgraphs
│ params:
│ r: 50
│ k: 4
│ i_level: 1
│ by_size: false
│ by_class: false
│ disjoint: true
│ - transform: SampleSegments
│ params:
│ ratio: 0.2
│ by_size: true
│ by_class: false
│ - transform: NAGRestrictSize
│ params:
│ level: 1+
│ num_nodes: 50000
│ - transform: NAGCast
│ - transform: NAGJitterKey
│ params:
│ key: pos
│ sigma: 0.05
│ trunc: 0.1
│ - transform: RandomTiltAndRotate
│ params:
│ phi: 0.1
│ theta: 180
│ - transform: RandomAnisotropicScale
│ params:
│ delta: 0.2
│ - transform: RandomAxisFlip
│ params:
│ p: 0.5
│ - transform: OnTheFlyHorizontalEdgeFeatures
│ params:
│ keys:
│ - mean_off
│ - std_off
│ - mean_dist
│ - angle_source
│ - angle_target
│ - centroid_dir
│ - centroid_dist
│ - normal_angle
│ - log_length
│ - log_surface
│ - log_volume
│ - log_size
│ use_mean_normal: false
│ - transform: OnTheFlyVerticalEdgeFeatures
│ params:
│ keys: []
│ use_mean_normal: false
│ - transform: SampleEdges
│ params:
│ level: 1+
│ n_min: -1
│ n_max: -1
│ - transform: NAGRestrictSize
│ params:
│ level: 1+
│ num_edges: 1000000
│ - transform: NAGAddKeysTo
│ params:
│ level: 0
│ keys:
│ - intensity
│ - linearity
│ - planarity
│ - scattering
│ - verticality
│ - elevation
│ to: x
│ - transform: NAGAddKeysTo
│ params:
│ level: 1+
│ keys: []
│ to: x
│ - transform: NAGJitterKey
│ params:
│ key: x
│ sigma: 0
│ trunc: 0
│ - transform: NAGJitterKey
│ params:
│ key: edge_attr
│ sigma: 0
│ trunc: 0
│ - transform: NAGJitterKey
│ params:
│ key: v_edge_attr
│ sigma: 0
│ trunc: 0
│ - transform: NAGDropoutColumns
│ params:
│ p: 0
│ key: x
│ inplace: true
│ to_mean: false
│ - transform: NAGDropoutColumns
│ params:
│ p: 0.3
│ key: edge_attr
│ inplace: true
│ to_mean: false
│ - transform: NAGDropoutColumns
│ params:
│ p: 0
│ key: v_edge_attr
│ inplace: true
│ to_mean: false
│ - transform: NAGDropoutRows
│ params:
│ p: 0
│ key: x
│ to_mean: false
│ - transform: NAGDropoutRows
│ params:
│ p: 0
│ key: edge_attr
│ to_mean: false
│ - transform: NAGDropoutRows
│ params:
│ p: 0
│ key: v_edge_attr
│ to_mean: false
│ - transform: NAGAddSelfLoops
│ - transform: OnTheFlyInstanceGraph
│ params:
│ level: -1
│ num_classes: 8
│ k_max: 30
│ radius: 0.1
│ on_device_val_transform:
│ - transform: NodeSize
│ - transform: SampleSubNodes
│ params:
│ low: 0
│ high: 1
│ n_min: 128
│ n_max: 256
│ - transform: NAGCast
│ - transform: OnTheFlyHorizontalEdgeFeatures
│ params:
│ keys:
│ - mean_off
│ - std_off
│ - mean_dist
│ - angle_source
│ - angle_target
│ - centroid_dir
│ - centroid_dist
│ - normal_angle
│ - log_length
│ - log_surface
│ - log_volume
│ - log_size
│ use_mean_normal: false
│ - transform: OnTheFlyVerticalEdgeFeatures
│ params:
│ keys: []
│ use_mean_normal: false
│ - transform: NAGAddKeysTo
│ params:
│ level: 0
│ keys:
│ - intensity
│ - linearity
│ - planarity
│ - scattering
│ - verticality
│ - elevation
│ to: x
│ - transform: NAGAddKeysTo
│ params:
│ level: 1+
│ keys: []
│ to: x
│ - transform: NAGAddSelfLoops
│ - transform: OnTheFlyInstanceGraph
│ params:
│ level: -1
│ num_classes: 8
│ k_max: 30
│ radius: 0.1
│ on_device_test_transform:
│ - transform: NodeSize
│ - transform: SampleSubNodes
│ params:
│ low: 0
│ high: 1
│ n_min: 128
│ n_max: 256
│ - transform: NAGCast
│ - transform: OnTheFlyHorizontalEdgeFeatures
│ params:
│ keys:
│ - mean_off
│ - std_off
│ - mean_dist
│ - angle_source
│ - angle_target
│ - centroid_dir
│ - centroid_dist
│ - normal_angle
│ - log_length
│ - log_surface
│ - log_volume
│ - log_size
│ use_mean_normal: false
│ - transform: OnTheFlyVerticalEdgeFeatures
│ params:
│ keys: []
│ use_mean_normal: false
│ - transform: NAGAddKeysTo
│ params:
│ level: 0
│ keys:
│ - intensity
│ - linearity
│ - planarity
│ - scattering
│ - verticality
│ - elevation
│ to: x
│ - transform: NAGAddKeysTo
│ params:
│ level: 1+
│ keys: []
│ to: x
│ - transform: NAGAddSelfLoops
│ - transform: OnTheFlyInstanceGraph
│ params:
│ level: -1
│ num_classes: 8
│ k_max: 30
│ radius: 0.1
│ tta_runs: null
│ tta_val: false
│ submit: false
│ dataloader:
│ batch_size: 4
│ num_workers: 4
│ pin_memory: true
│ persistent_workers: true
│ trainval: true
│ val_on_test: true
│ xy_tiling: 3
│ voxel: 0.1
│ knn: 25
│ knn_r: 10
│ knn_step: -1
│ knn_min_search: 10
│ ground_threshold: 5
│ ground_scale: 20
│ pcp_regularization:
│ - 0.1
│ - 0.2
│ - 0.3
│ pcp_spatial_weight:
│ - 0.1
│ - 0.01
│ - 0.001
│ pcp_cutoff:
│ - 10
│ - 30
│ - 100
│ pcp_k_adjacency: 10
│ pcp_w_adjacency: 1
│ pcp_iterations: 15
│ graph_k_min: 1
│ graph_k_max: 30
│ graph_gap:
│ - 5
│ - 30
│ - 30
│ graph_se_ratio: 0.3
│ graph_se_min: 20
│ graph_cycles: 3
│ graph_margin: 0.5
│ graph_chunk:
│ - 1000000.0
│ - 100000.0
│ - 100000.0
│ sample_segment_ratio: 0.2
│ sample_segment_by_size: true
│ sample_segment_by_class: false
│ sample_point_min: 32
│ sample_point_max: 128
│ sample_graph_r: 50
│ sample_graph_k: 4
│ sample_graph_disjoint: true
│ sample_edge_n_min: -1
│ sample_edge_n_max: -1
│ pos_jitter: 0.05
│ tilt_n_rotate_phi: 0.1
│ tilt_n_rotate_theta: 180
│ anisotropic_scaling: 0.2
│ node_feat_jitter: 0
│ h_edge_feat_jitter: 0
│ v_edge_feat_jitter: 0
│ node_feat_drop: 0
│ h_edge_feat_drop: 0.3
│ v_edge_feat_drop: 0
│ node_row_drop: 0
│ h_edge_row_drop: 0
│ v_edge_row_drop: 0
│ drop_to_mean: false
│
├── model
│ └── _target_: src.models.semantic.SemanticSegmentationModule
│ num_classes: 8
│ sampling_loss: false
│ loss_type: ce_kl
│ weighted_loss: true
│ init_linear: null
│ init_rpe: null
│ multi_stage_loss_lambdas:
│ - 1
│ - 50
│ transformer_lr_scale: 0.1
│ gc_every_n_steps: 0
│ optimizer:
│ _target_: torch.optim.AdamW
│ _partial_: true
│ lr: 0.01
│ weight_decay: 0.0001
│ scheduler:
│ _target_: src.optim.CosineAnnealingLRWithWarmup
│ _partial_: true
│ T_max: 380
│ eta_min: 1.0e-06
│ warmup_init_lr: 1.0e-06
│ num_warmup: 20
│ warmup_strategy: cos
│ criterion:
│ _target_: torch.nn.CrossEntropyLoss
│ ignore_index: 8
│ _point_mlp:
│ - 32
│ - 64
│ - 128
│ _node_mlp_out: 32
│ _h_edge_mlp_out: 32
│ _v_edge_mlp_out: 32
│ _point_hf_dim: 10
│ _node_hf_dim: 0
│ _node_injection_dim: 4
│ _h_edge_hf_dim: 18
│ _v_edge_hf_dim: 0
│ _down_dim:
│ - 64
│ - 64
│ - 64
│ - 64
│ _up_dim:
│ - 64
│ - 64
│ - 64
│ _mlp_depth: 2
│ net:
│ point_mlp:
│ - 10
│ - 32
│ - 64
│ - 128
│ point_drop: null
│ down_dim:
│ - 64
│ - 64
│ down_pool_dim:
│ - 128
│ - 64
│ - 64
│ - 64
│ down_in_mlp:
│ - - 132
│ - 64
│ - 64
│ - - 68
│ - 64
│ - 64
│ down_out_mlp: null
│ down_mlp_drop: null
│ down_num_heads: 16
│ down_num_blocks: 3
│ down_ffn_ratio: 1
│ down_residual_drop: null
│ down_attn_drop: null
│ down_drop_path: null
│ up_dim:
│ - 64
│ up_in_mlp:
│ - - 132
│ - 64
│ - 64
│ up_out_mlp: null
│ up_mlp_drop: null
│ up_num_heads: 16
│ up_num_blocks: 1
│ up_ffn_ratio: 1
│ up_residual_drop: null
│ up_attn_drop: null
│ up_drop_path: null
│ activation:
│ _target_: torch.nn.LeakyReLU
│ norm:
│ _target_: src.nn.GraphNorm
│ _partial_: true
│ pre_norm: true
│ no_sa: false
│ no_ffn: true
│ qk_dim: 4
│ qkv_bias: true
│ qk_scale: null
│ in_rpe_dim: 32
│ k_rpe: true
│ q_rpe: true
│ v_rpe: true
│ k_delta_rpe: false
│ q_delta_rpe: false
│ qk_share_rpe: false
│ q_on_minus_rpe: false
│ stages_share_rpe: false
│ blocks_share_rpe: false
│ heads_share_rpe: false
│ _target_: src.models.components.spt.SPT
│ nano: false
│ node_mlp: null
│ h_edge_mlp:
│ - 18
│ - 32
│ - 32
│ v_edge_mlp:
│ - 0
│ - 32
│ - 32
│ share_hf_mlps: false
│ mlp_activation:
│ _target_: torch.nn.LeakyReLU
│ mlp_norm:
│ _target_: src.nn.GraphNorm
│ _partial_: true
│ use_pos: true
│ use_node_hf: true
│ use_diameter: false
│ use_diameter_parent: true
│ pool: max
│ unpool: index
│ fusion: cat
│ norm_mode: graph
│
├── logger
│ └── wandb:
│ project: spt_dales
│ name: SPT-64
│
├── trainer
│ └── _target_: pytorch_lightning.Trainer
│ default_root_dir: superpoint_transformer/logs/eval/runs/2024-03-06_14-28-08
│ min_epochs: 1
│ max_epochs: 400
│ accelerator: gpu
│ devices: 1
│ check_val_every_n_epoch: 10
│ deterministic: false
│ precision: 32
│
├── paths
│ └── root_dir:superpoint_transformer
│ data_dir: superpoint_transformer/data/
│ log_dir: superpoint_transformer/logs/
│ output_dir: superpoint_transformer/logs/eval/runs/2024-03-06_14-28-08
│ work_dir: superpoint_transformer
│
├── extras
│ └── ignore_warnings: false
│ enforce_tags: true
│ print_config: true
│
├── task_name
│ └── eval
├── tags
│ └── ['dev']
├── compile
│ └── False
├── ckpt_path
│ └── models/spt-2_dales.ckpt
└── float32_matmul_precision
└── high
[2024-03-06 14:28:09,279][__main__][INFO] - Instantiating datamodule <src.datamodules.dales.DALESDataModule>
[2024-03-06 14:28:09,455][__main__][INFO] - Instantiating model <src.models.semantic.SemanticSegmentationModule>
[2024-03-06 14:28:09,530][__main__][INFO] - Instantiating loggers...
[2024-03-06 14:28:09,530][__main__][INFO] - Instantiating trainer <pytorch_lightning.Trainer>
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
[2024-03-06 14:28:09,596][__main__][INFO] - Starting testing!
Processing...
/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/deprecation.py:26: UserWarning: 'makedirs' is deprecated, use 'os.makedirs(path, exist_ok=True)' instead
warnings.warn(out)
0%| | 0/261 [00:00<?, ?it/s]
[2024-03-06 14:28:10,570][src.utils.utils][ERROR] -
Traceback (most recent call last):
File "/home/dev/project/segmentation/code/superpoint_transformer/src/utils/utils.py", line 45, in wrap
metric_dict, object_dict = task_func(cfg=cfg)
File "src/eval.py", line 105, in evaluate
trainer.test(model=model, datamodule=datamodule, ckpt_path=cfg.ckpt_path)
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 754, in test
return call._call_and_handle_interrupt(
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 44, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 794, in _test_impl
results = self._run(model, ckpt_path=ckpt_path)
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 947, in _run
self._data_connector.prepare_data()
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py", line 94, in prepare_data
call._call_lightning_datamodule_hook(trainer, "prepare_data")
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 179, in _call_lightning_datamodule_hook
return fn(*args, **kwargs)
File "/home/dev/project/segmentation/code/superpoint_transformer/src/datamodules/base.py", line 144, in prepare_data
self.dataset_class(
File "/home/dev/project/segmentation/code/superpoint_transformer/src/datasets/base.py", line 223, in __init__
super().__init__(root, transform, pre_transform, pre_filter)
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/data/in_memory_dataset.py", line 82, in __init__
super().__init__(root, transform, pre_transform, pre_filter, log,
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/data/dataset.py", line 118, in __init__
self._process()
File "/home/dev/project/segmentation/code/superpoint_transformer/src/datasets/base.py", line 647, in _process
self.process()
File "/home/dev/project/segmentation/code/superpoint_transformer/src/datasets/base.py", line 682, in process
self._process_single_cloud(p)
File "/home/dev/project/segmentation/code/superpoint_transformer/src/datasets/base.py", line 703, in _process_single_cloud
data = SampleXYTiling(x=tile[0], y=tile[1], tiling=tile[2])(data)
File "/home/dev/project/segmentation/code/superpoint_transformer/src/transforms/transforms.py", line 23, in __call__
return self._process(x)
File "/home/dev/project/segmentation/code/superpoint_transformer/src/transforms/sampling.py", line 372, in _process
return data.select(idx)[0]
File "/home/dev/project/segmentation/code/superpoint_transformer/src/data/data.py", line 396, in select
if is_tensor and is_node_size and key in self.v_edge_keys:
File "/home/dev/project/segmentation/code/superpoint_transformer/src/data/data.py", line 150, in v_edge_keys
return [k for k in self.keys if k.startswith('v_edge_')]
TypeError: 'method' object is not iterable
[2024-03-06 14:28:10,571][src.utils.utils][INFO] - Closing loggers...
Error executing job with overrides: ['experiment=semantic/dales', 'ckpt_path=models/spt-2_dales.ckpt']
Traceback (most recent call last):
File "src/eval.py", line 117, in main
evaluate(cfg)
File "/home/dev/project/segmentation/code/superpoint_transformer/src/utils/utils.py", line 48, in wrap
raise ex
File "/home/dev/project/segmentation/code/superpoint_transformer/src/utils/utils.py", line 45, in wrap
metric_dict, object_dict = task_func(cfg=cfg)
File "src/eval.py", line 105, in evaluate
trainer.test(model=model, datamodule=datamodule, ckpt_path=cfg.ckpt_path)
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 754, in test
return call._call_and_handle_interrupt(
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 44, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 794, in _test_impl
results = self._run(model, ckpt_path=ckpt_path)
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 947, in _run
self._data_connector.prepare_data()
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py", line 94, in prepare_data
call._call_lightning_datamodule_hook(trainer, "prepare_data")
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 179, in _call_lightning_datamodule_hook
return fn(*args, **kwargs)
File "/home/dev/project/segmentation/code/superpoint_transformer/src/datamodules/base.py", line 144, in prepare_data
self.dataset_class(
File "/home/dev/project/segmentation/code/superpoint_transformer/src/datasets/base.py", line 223, in __init__
super().__init__(root, transform, pre_transform, pre_filter)
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/data/in_memory_dataset.py", line 82, in __init__
super().__init__(root, transform, pre_transform, pre_filter, log,
File "/home/dev/anaconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/data/dataset.py", line 118, in __init__
self._process()
File "/home/dev/project/segmentation/code/superpoint_transformer/src/datasets/base.py", line 647, in _process
self.process()
File "/home/dev/project/segmentation/code/superpoint_transformer/src/datasets/base.py", line 682, in process
self._process_single_cloud(p)
File "/home/dev/project/segmentation/code/superpoint_transformer/src/datasets/base.py", line 703, in _process_single_cloud
data = SampleXYTiling(x=tile[0], y=tile[1], tiling=tile[2])(data)
File "/home/dev/project/segmentation/code/superpoint_transformer/src/transforms/transforms.py", line 23, in __call__
return self._process(x)
File "/home/dev/project/segmentation/code/superpoint_transformer/src/transforms/sampling.py", line 372, in _process
return data.select(idx)[0]
File "/home/dev/project/segmentation/code/superpoint_transformer/src/data/data.py", line 396, in select
if is_tensor and is_node_size and key in self.v_edge_keys:
File "/home/dev/project/segmentation/code/superpoint_transformer/src/data/data.py", line 150, in v_edge_keys
return [k for k in self.keys if k.startswith('v_edge_')]
TypeError: 'method' object is not iterable
Looking forward to your replay!
better1593 commented
Hi, @QingXia1994! I had the same problem when using torch_geometric 2.5.0. I resolved it by downgrading to version 2.3.0. You may want to check out my previous comment.
Hi Damien, I tried to change torch version 2.3.0 into 2.0.0 and it worked. But when I tried to run the command python src/train.py experiment=semantic/dales, I received a prompt saying I didn't insatll the torch_geometric module. So, I directly installed torch_geometric using pip install torch_geometric and attempted to run python src/train.py experiment=semantic/dales again. This time, I encountered an error as follows:
Processing... /root/miniconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/deprecation.py:26: UserWarning: 'makedirs' is deprecated, use 'os.makedirs(path, exist_ok=True)' instead warnings.warn(out) 0%| | 0/261 [00:05<?, ?it/s] [�[36m2024-03-02 12:56:24,511�[39m][�[34msrc.utils.utils�[39m][�[31mERROR�[39m] - Traceback (most recent call last): ... File "/root/autodl-tmp/superpoint_transformer/src/data/data.py", line 150, in v_edge_keys return [k for k in self.keys if k.startswith('v_edge_')] TypeError: 'method' object is not iterableI suspect it's due to the version of torch_geomtric, so I rolled back to version 2.3.0 and the problem was resolved.
QingXia1994 commented
@better1593 It works when i change the PyG from version 2.5.0 to 2.3.0. Thanks for your replay!
drprojects commented
Thanks a lot @better1593 for your help 🙏
drprojects commented
The latest commit modifies the install.sh script to explicitly install torch_geometric==2.3.0. This should solve some of the issues mentioned here. The main problem is FRNN installation, for which I cannot provide support. I invite you to create an issue on the relevant repository.
A workaround for the FRNN compilation issue has been proposed by @noisyneighbour here. On the longer run, FRNN would ideally be updated, or this little hack would be automated in install.sh.