multimodallearning/pytorch-mask-rcnn

Get some warning while compiling nms

sjytker opened this issue · 4 comments

warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
unsigned long long* mask_flat = THCudaLongTensor_data(state, mask);
^
/home/sujunyu/planercnn-master/nms/src/nms_cuda.c:37:40: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
unsigned long long * mask_cpu_flat = THLongTensor_data(mask_cpu);
^
/home/sujunyu/planercnn-master/nms/src/nms_cuda.c:40:39: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
unsigned long long* remv_cpu_flat = THLongTensor_data(remv_cpu);
^
/home/sujunyu/planercnn-master/nms/src/nms_cuda.c:23:7: warning: unused variable 'boxes_dim' [-Wunused-variable]
int boxes_dim = THCudaTensor_size(state, boxes, 1);

These lead to the problem " can't find nms.gpu_nms " while running the programme. I assume it is due to the unsuccessfully compilation of nms.

I've already used docker with gcc version 5.4, cuda 9.0

solved by modifying unsigned long long into int64_t

@Suujy in which files do u change unsigned long long to int?

@Suujy in which files do u change unsigned long long to int?

I don't remember. From the warning above, maybe it's nms_cuda.c?

Worked, also had to change the long longs to int64_t in:
nms_kernel.cu
nms_kernel.h
thanks!