ddshan/hand_object_detector

Error AtomicAdd running python setup.py build develop

Opened this issue · 2 comments

I am trying to build faster_rcnn with ```python setup.py build develop``. I am getting this error (every time atomicAdd is used):

error: no instance of overloaded function "atomicAdd" matches the argument list
            argument types are: (double *, double)
          detected during instantiation of "void RoIAlignBackwardFeature(int, const T *, int, T, int, int, int, int, int, int, T *, const T *) [with T=double]" 

The final error log is:

Traceback (most recent call last):
  File "setup.py", line 59, in <module>
    setup(
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/__init__.py", line 107, in setup
    return distutils.core.setup(**attrs)
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
    self.run_command(cmd_name)
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 84, in run
    _build_ext.run(self)
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
    self.build_extensions()
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 765, in build_extensions
    build_ext.build_extensions(self)
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
    self._build_extensions_serial()
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
    self.build_extension(ext)
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
    _build_ext.build_extension(self, ext)
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
    objects = self.compiler.compile(
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 586, in unix_wrap_ninja_compile
    _write_ninja_file_and_compile_objects(
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1487, in _write_ninja_file_and_compile_objects
    _run_ninja_build(
  File "/itet-stor/sruffino/net_scratch/conda_envs/handobjdet/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension```

I had the same environment as you mentioned, one possible issue could be with CUDA driver.
Do you have any idea on how to fix it?

PS: it builds perfectly when I am not using GPU

I figured out that, since my coda capability is 5.0 at maximum, atomAdd is not supported (it is from 6.0 on) and I had to define it by myself. See https://stackoverflow.com/questions/37566987/cuda-atomicadd-for-doubles-definition-error for more explanation. In particular, I had to add at the beginning of RoiAlign_cuda.cu and RoiPool_cuda.cu the following lines of code:

/ Define custom atomicAdd when cuda capability is less then 6.00
#if __CUDA_ARCH__ < 600
__device__ double myatomicAdd2(double* address, double val)
{
    unsigned long long int* address_as_ull =
                              (unsigned long long int*)address;
    unsigned long long int old = *address_as_ull, assumed;

    do {
        assumed = old;
        old = atomicCAS(address_as_ull, assumed,
                        __double_as_longlong(val +
                               __longlong_as_double(assumed)));

    // Note: uses integer comparison to avoid hang in case of NaN (since NaN != NaN)
    } while (assumed != old);

    return __longlong_as_double(old);
}
#endif
```.
Moreover, pay attention that you should define them with different names (e.g. 1 and 2 for example) or define it in another file as global function.

Hi @samueleruffino99

i followed your suggestion but still getting below error :
any help ?

error: argument of type "float *" is incompatible with parameter of type "double *"

image

thanks in advance