How can l get the features vector associated to each superpixel ?

[pinkfloyd06]

Hello,

Thank you a lot for your work.

I'm wondering your network learns also features vector of each superpixel or it just learn the superpixels.
If yes, how can l get them otherwise you keep just the raw pixels of each superpixels.

Thank you

[varunjampani]

Our network first learns deep pixel features which are then used for computing distance between pixels and superpixels. We do iterative clustering using deep features. After each iteration of differentiable SLIC, we obtain superpixel centroid features. You can use the following function to convert deep pixel features into superpixel features:

spixel_feat = L.SpixelFeature2(pixel_features,
                                                    pixel_assoc,
                                                    spixel_init,
                                                    spixel_feature2_param =\
        dict(num_spixels_h = num_spixels_h, num_spixels_w = num_spixels_w))

Does this answer your question?

[pinkfloyd06]

Hi @varunjampani ,

Thank you for your answer. Yes this is what l need. Can l also get the coordinates of each superpixel (gravity center) and the coordinates (index(x,y)) of each pixel ?

Thank you

[varunjampani]

You can compute 'XYRGB' pixel features using the following layer:

pixel_features = L.PixelFeature(img,
                                      pixel_feature_param = dict(type = P.PixelFeature.POSITION_AND_RGB,
                                                                 pos_scale = float(pos_scale),
color_scale = float(color_scale)))

And, then pass these pixels features onto 'SpixelFeature2' layer to compute XYRGB gravity center for superpixels:

spixel_feat = L.SpixelFeature2(pixel_features,
                                                    pixel_assoc,
                                                    spixel_init,
                                                    spixel_feature2_param =\
        dict(num_spixels_h = num_spixels_h, num_spixels_w = num_spixels_w))

And, you can convert pixel-superpixel associations into 'superpixel index' using the following function in 'create_net.py':

def compute_final_spixel_labels(pixel_spixel_assoc,
                                spixel_init,
                                num_spixels_h, num_spixels_w):

    # Compute new spixel indices
    rel_label = L.ArgMax(pixel_spixel_assoc, argmax_param = dict(axis = 1),
                         propagate_down = False)
    new_spix_indices = L.RelToAbsIndex(rel_label, spixel_init,
                                       rel_to_abs_index_param = dict(num_spixels_h = int(num_spixels_h),
                                                                     num_spixels_w = int(num_spixels_w)),
                                                                     propagate_down = [False, False])

    return new_spix_indices