Empty list on Scatter_nearest_neighbor
billy-antoine opened this issue · 10 comments
Hello,
First of all, thank you for your amazing work.
I am trying to train the model on my own data. I am running on WSL Ubuntu 22.04.4, and the installation went well.
I have sorted my point clouds into several folders (train, val and test) and created a datamodule.yaml, a config.py and a dataset.py based on dales templates. My point clouds are in .laz format so I just made a little change for the read_tile function.
I have an error in the scatter_nearest_neighbor function that I can't fix. It seems that they don't find any neighbor to a given point :
File "/home/abilly/dev/superpoint_transformer/src/utils/scatter.py", line 138, in scatter_nearest_neighbor candidate = torch.cat([elt[0] for elt in out_list], dim=0) RuntimeError: torch.cat(): expected a non-empty list of Tensors
I cleaned my point clouds to remove isolated points but this didn't seem to change anyting..
I saw that #36 suggested to modify parameters in the .yaml, but I'm not sure of the one I can play with.
Do you think you can help me?
Don't hesitate if you need more info.
Hi @billy-antoine, thanks for your interest in our project.
A bit of context: scatter_nearest_neighbor
is not computing distance between points, but between superpoints. From what I see, the error comes from the fact that your edge_index
is empty.
I am guessing this error occurs at preprocessing time when calling RadiusHorizontalGraph
? If so, have you tried increasing graph_gap
as described in #36 ?
PS: If you ❤️ or use this project, don't forget to give it a ⭐, it means a lot to us !
Hi @drprojects, thank you for your quick answer!
You are correct, the error occurs during the preprocessing, here is the full log if it might help you understand :
Error executing job with overrides: ['experiment=semantic/viamapa']
Traceback (most recent call last):
File "/home/abilly/dev/superpoint_transformer/src/train.py", line 139, in main
metric_dict, _ = train(cfg)
File "/home/abilly/dev/superpoint_transformer/src/utils/utils.py", line 48, in wrap
raise ex
File "/home/abilly/dev/superpoint_transformer/src/utils/utils.py", line 45, in wrap
metric_dict, object_dict = task_func(cfg=cfg)
File "/home/abilly/dev/superpoint_transformer/src/train.py", line 114, in train
trainer.fit(model=model, datamodule=datamodule, ckpt_path=cfg.get("ckpt_path"))
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 544, in fit
call._call_and_handle_interrupt(
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 44, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 580, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 947, in _run
self._data_connector.prepare_data()
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py", line 94, in prepare_data
call._call_lightning_datamodule_hook(trainer, "prepare_data")
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 179, in _call_lightning_datamodule_hook
return fn(*args, **kwargs)
File "/home/abilly/dev/superpoint_transformer/src/datamodules/base.py", line 144, in prepare_data
self.dataset_class(
File "/home/abilly/dev/superpoint_transformer/src/datasets/base.py", line 223, in __init__
super().__init__(root, transform, pre_transform, pre_filter)
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/data/in_memory_dataset.py", line 57, in __init__
super().__init__(root, transform, pre_transform, pre_filter, log)
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/data/dataset.py", line 97, in __init__
self._process()
File "/home/abilly/dev/superpoint_transformer/src/datasets/base.py", line 647, in _process
self.process()
File "/home/abilly/dev/superpoint_transformer/src/datasets/base.py", line 682, in process
self._process_single_cloud(p)
File "/home/abilly/dev/superpoint_transformer/src/datasets/base.py", line 710, in _process_single_cloud
nag = self.pre_transform(data)
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/transforms/compose.py", line 24, in __call__
data = transform(data)
File "/home/abilly/dev/superpoint_transformer/src/transforms/transforms.py", line 23, in __call__
return self._process(x)
File "/home/abilly/dev/superpoint_transformer/src/transforms/graph.py", line 656, in _process
nag = _horizontal_graph_by_radius(
File "/home/abilly/dev/superpoint_transformer/src/transforms/graph.py", line 760, in _horizontal_graph_by_radius
nag = _horizontal_graph_by_radius_for_single_level(
File "/home/abilly/dev/superpoint_transformer/src/transforms/graph.py", line 814, in _horizontal_graph_by_radius_for_single_level
edge_index, distances = cluster_radius_nn_graph(
File "/home/abilly/dev/superpoint_transformer/src/utils/neighbors.py", line 488, in cluster_radius_nn_graph
anchors = scatter_nearest_neighbor(
File "/home/abilly/dev/superpoint_transformer/src/utils/scatter.py", line 138, in scatter_nearest_neighbor
candidate = torch.cat([elt[0] for elt in out_list], dim=0)
RuntimeError: torch.cat(): expected a non-empty list of Tensors
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
I indeed tried to play with graph_gap, but I'm not sure how much can I increase the values. I went to 10 times the original bvalues but the result is still the same. My clouds are quite sparse as they are acquired from satellites :
I gave my best star, thanks for the reminder ⭐
Maybe your point density is much lower than for DALES and your hierarchical superpoint partitions are not so good.
Have you tried visualizing your superpoint partitions ?
- temporary hack for the sake investigation: remove
RadiusHorizontalGraph
from your datamodule's prepreprocessing - use our visualization tool to inspect each partition level (see our demo notebooks for instance)
As a rule of thumb, you should parameterize the partition so that
Also, you want to be wary of clouds with only 1 superpoint in
From there, you should further investigate which cloud causes the error you are encountering
- identify the file / cloud id of the cloud when preprocessing crashes
- in a notebook or whatever environment you work with, load the corresponding preprocessed
NAG
(from the previous hack where you skippedRadiusHorizontalGraph
) - try manually applying various
RadiusHorizontalGraph
graph_gap
parametrizations and find out why youredge_index
in here is empty
Hi again,
Removing RadiusHorizontalGraph allowed me to go further into the process, but it still crashes before the preprocessing finishes. The error occurs while calling grid_cluster dring the voxelization by GridSampling3D.
No matter what point cloud I use, there is always a time during the voxelisation where my data seems to be empty :
This generates the following error:
Processing...
11%|█ | 2/18 [00:47<06:16, 23.51s/it]
[2024-06-05 16:11:46,262][src.utils.utils][ERROR] -
Traceback (most recent call last):
File "/home/abilly/dev/superpoint_transformer/src/utils/utils.py", line 45, in wrap
metric_dict, object_dict = task_func(cfg=cfg)
File "/home/abilly/dev/superpoint_transformer/src/train.py", line 114, in train
trainer.fit(model=model, datamodule=datamodule, ckpt_path=cfg.get("ckpt_path"))
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 544, in fit
call._call_and_handle_interrupt(
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 44, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 580, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 947, in _run
self._data_connector.prepare_data()
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py", line 94, in prepare_data
call._call_lightning_datamodule_hook(trainer, "prepare_data")
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 179, in _call_lightning_datamodule_hook
return fn(*args, **kwargs)
File "/home/abilly/dev/superpoint_transformer/src/datamodules/base.py", line 144, in prepare_data
self.dataset_class(
File "/home/abilly/dev/superpoint_transformer/src/datasets/base.py", line 223, in __init__
super().__init__(root, transform, pre_transform, pre_filter)
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/data/in_memory_dataset.py", line 57, in __init__
super().__init__(root, transform, pre_transform, pre_filter, log)
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/data/dataset.py", line 97, in __init__
self._process()
File "/home/abilly/dev/superpoint_transformer/src/datasets/base.py", line 647, in _process
self.process()
File "/home/abilly/dev/superpoint_transformer/src/datasets/base.py", line 682, in process
self._process_single_cloud(p)
File "/home/abilly/dev/superpoint_transformer/src/datasets/base.py", line 710, in _process_single_cloud
nag = self.pre_transform(data)
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/torch_geometric/transforms/compose.py", line 24, in __call__
data = transform(data)
File "/home/abilly/dev/superpoint_transformer/src/transforms/transforms.py", line 23, in __call__
return self._process(x)
File "/home/abilly/dev/superpoint_transformer/src/transforms/sampling.py", line 151, in _process
cluster = grid_cluster(coords, torch.ones(3, device=coords.device))
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/torch_cluster/grid.py", line 34, in grid_cluster
return torch.ops.torch_cluster.grid(pos, size, start, end)
File "/home/abilly/miniconda3/envs/spt/lib/python3.8/site-packages/torch/_ops.py", line 755, in __call__
return self._op(*args, **(kwargs or {}))
RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1] because the unspecified dimension size -1 can be any value and is ambiguous
I have seen the visualization tool, it works fine on the demo data, but I have the same error with my cloud during the preprocessing, which does not allow me to generate the NAG and look at my superpoint partitions...
Thank you again for your time, I hope I will be able to make it work without bothering you too much!
This means that your point cloud is empty, this is not a GridSampling3D
error. You need to thoroughly investigate that all your point clouds contain points. In particular, check the output of read_single_raw_cloud()
for all your datasets.
Hi again,
Thank you once more for your time.
None of my clouds seems to be empty. My dataset is very small so it is easy to check.
The read_single_raw_cloud() seems to work fine as it returns the following before crashing:
Data(pos=[2660907, 3], pos_offset=[3], intensity=[2660907], y=[2660907], obj=InstanceData(num_clusters=2660907, num_overlaps=2660907, num_obj=1, device=cpu))
I tried with multiple clouds, even with a whole new dataset with more classical points architecture and the error remains the same.
Here is an example of a cloud that generates the error:
What is your voxel
size ? Can you try isolating (ie Data.save()
somewhere to disk) the problematic cloud's Data
object and Data.load()
it in a script/notebook to simply run GridSampling3D
on it ?
I ran into the same error as you, and it turned out to be related to the data. Try to print out the shapes of your nag attributes to inspect how many superpoints there are at various points of the transform chain in the 3rd hierarchy level especially.
@billy-antoine have you investigated your point cloud and partition sizes and solved your issue ? May I close it ?
I am considering this issue solved. @billy-antoine feel free to reopen it if your problem persists after investigation of your data.