How does the trilinear interpolation work if the dimensions don't match?

[jayroopramesh]

Hey, i am stuck with this error at the function: def trilinear_interp(self, x, voxel_min_vertex, voxel_max_vertex, voxel_embedds) in class HashEmbedder(nn.Module):

RuntimeError: The size of tensor a (2) must match the size of tensor b (3) at non-singleton dimension 2
Emerging at the following line:

c00 = voxel_embedds[:,0]*(1-weights[:,0][:,None]) + voxel_embedds[:,4]*weights[:,0][:,None]

My input was a 3D coordinate, [X, Y, Z], and I do not see where I can reshape or change, because its the voxel embedding that is the problem:

Considering an input with 3 dimensions, and the hash embedding having 2 dimensions, how does this even work? The following is the specs in the function itself:

• x: B x 3
• voxel_min_vertex: B x 3
• voxel_max_vertex: B x 3
• voxel_embedds: B x 8 x 2
• weights = (x - voxel_min_vertex)/(voxel_max_vertex-voxel_min_vertex) # B x 3

Please help!

[yashbhalgat]

Hi, please check if the shape of your input x is indeed (B, 3) (in your case, maybe (1, 3)?) or is it just (3, )?

[jayroopramesh]

Hi, thank you for quick response!

Yes, after checking shape is indeed (25600, 3) for x input, after flattening original size of (160, 160, 3). My question, the Hash Table, i.e, voxel_embedds, is always of size (25600, 2), which makes sense because as per the paper n_features_per_level=2. So, I am not sure how to get B x 8 x 2 with the voxel embedding! Please assist! :)

[jayroopramesh]

Also, this code might seem to solve the issue (I suspect, still checking it out), why was this commented out btw?

  # hashed_voxel_indices = [] # B x 8 ... 000,001,010,011,100,101,110,111
    # for i in [0, 1]:
    #     for j in [0, 1]:
    #         for k in [0, 1]:
    #             vertex_idx = bottom_left_idx + torch.tensor([i,j,k])
    #             # vertex = bottom_left + torch.tensor([i,j,k])*grid_size
    #             hashed_voxel_indices.append(hash(vertex_idx, log2_hashmap_size))```

[jayroopramesh]

Hey! I think I got it, my mistake, I somehow did not copy the Box-Offsets. Although, I wish to ask about the following lines: what are 0 and 1 indicating?

BOX_OFFSETS = torch.tensor([[[i,j,k] for i in [0, 1] for j in [0, 1] for k in [0, 1]]], device='cuda')

Secondly, the default minimum and maximum bounds are [100,100,100] and [-100,-100,-100]. How can I decide this number? For instance, I have a volume that is 160 x 160 x 160, with no ray tracing or density components. Therefore, would my bounds be [0,0,0] and [-160,-160,-160]?

Correct if I am wrong, but the Total Variation Loss, is for counting for each embedding hash layer right?