Questions about parametrization

[noisyneighbour]

Hello!

First off, thank you for your work on this project.

I am trying to use SP graph clustering on my own forestry dataset consisting of tree instances and I'm running into difficulties with the instance predictions. In the input my dataset has 2 stuff classes: ground and "other" and one thing class: tree canopies + tree trunks. Each tree is its own instance and they are usually around 20-30m high. In the results however, I am finding that tree trunks are often a separate instance from the tree canopy and the tree canopies are often times split into many instances.

I tried experimenting with various parameters, but mostly achieved similar results. These are some parameters I've experimented with trying to resolve the issue:

pcp_cutoff="[1,5,100],[1,10,100],[1,15,100],[1,20,100]"
pcp_spatial_weight="[1,1e-1,1e-2],[1,1e-2,1e-3]"
instance_k_max=5,10,15
instance_radius=5,10,20,30

Parameters that achieved the best results so far:

voxel: 0.1
knn: 25
knn_r: 15
knn_step: -1
knn_min_search: 10
ground_threshold: 65
percentile_threshold: 1
ground_scale: 20
pcp_regularization: [0.1, 0.2, 0.3]
pcp_spatial_weight: [1e-1, 1e-2, 1e-3]
pcp_cutoff: [1, 5, 100]
pcp_k_adjacency: 10
pcp_w_adjacency: 1
pcp_iterations: 15
graph_k_min: 1
graph_k_max: 30
graph_gap: [5, 30, 30]
graph_se_ratio: 0.3
graph_se_min: 20
graph_cycles: 3
graph_margin: 0.5
graph_chunk: [1e6, 1e5, 1e5]  # reduce if CUDA memory errors

# Batch construction parameterization
sample_segment_ratio: 0.2
sample_segment_by_size: True
sample_segment_by_class: False
sample_point_min: 32
sample_point_max: 128
sample_graph_r: 50  # set to r<=0 to skip SampleRadiusSubgraphs
sample_graph_k: 4
sample_graph_disjoint: True
sample_edge_n_min: -1  # [5, 5, 15]
sample_edge_n_max: -1  # [10, 15, 25]

# Augmentations parameterization
pos_jitter: 0.05
tilt_n_rotate_phi: 0.1
tilt_n_rotate_theta: 180
anisotropic_scaling: 0.2
node_feat_jitter: 0
h_edge_feat_jitter: 0
v_edge_feat_jitter: 0
node_feat_drop: 0
h_edge_feat_drop: 0.0
v_edge_feat_drop: 0
node_row_drop: 0
h_edge_row_drop: 0
v_edge_row_drop: 0
drop_to_mean: False

Maybe somebody here can help me by pointing out which parameters should I focus my experiments on to resolve this issue?

[drprojects]

It seems your $P_1$ and $P_2$ partitions superpoints may be a bit small. In particular, you may want to try to increase the average number of points per superpoint $\frac{|P_0|}{|P_1|}$. A rule of thumb we tried to follow in our datasets (this is no golden rule) is $\frac{|P_0|}{|P_1|} \sim 30$ and $\frac{|P_1|}{|P_2|} \sim 5$.

To adjust your $P_1$ superpoint sizes, you can play with

• pcp_regularization[0]
• pcp_spatial_weight[0]
• pcp_cutoff[0]

Then, to adjust the panoptic segmentation performance, I would play with instance_radius as you seem to have already tried. This should rule the complexity of the superpoint graph on which the instances are computed. Increasing instance_radius should hopefully ensure that all trunk superpoints are properly connected to the associated canopy superpoints.

If this does not suffice, you may also want to play with the partitioner's parameters too...

Good luck !

PS: if you are working on open forestry data, we would be happy to expand the current codebase with your dataset if you send us a pull request 😉

[noisyneighbour]

Thank you very much! I'll play around with the parameters and report if it resolves the issue. Unfortunately this specific dataset is not open source, but I'll try to contribute in another way.

On a side note, currently the algorithm expects each instance to have only 1 unique classification. For my use case I would like to train on a dataset that has 2 unique classes in each instance. What do you think would be the best way to get rid of this restriction and allow multiple classes in each instance? I know this goes slightly beyond the definition of panoptic segmentation.

[drprojects]

🤔 for your multi-class setup, modifying the label manipulation logic all through the project might be quite involved. A simpler workaround would be to convert your label pairs into unique labels and treat these as your labels. Typically I would do something like:

def label_pair_to_pair_label(y1, y2, num_classes_1):
	"""
	y1 in ⟦0, num_classes_1⟦
	y2 in ⟦0, num_classes_2⟦

	y in ⟦0, num_classes_1 * num_classes_2⟦
	"""
	y = y1 + y2 * num_classes_1
	return y

def pair_label_to_label_pair(y, num_classes_1):
	"""
	y in ⟦0, num_classes_1 * num_classes_2⟦

	y1 in ⟦0, num_classes_1⟦
	y2 in ⟦0, num_classes_2⟦
	"""
	y1 = y % num_classes_1
	y2 = y // num_classes_1
	return y1, y2

This would allow preserving the current pipeline for labels and you can recover your multi-class labels at loss computation time if need be. Up to you on how you ask your model to predict multiple classes and how you supervise it, though. You would also need to investigate closer if/how this strategy affects the panoptic metrics computation. I cannot provide much more help in this direction, however, since this departs from our supported setting.