ap值相差很大

Question

ap值相差很大

Closed this issue 5 years ago · 10 comments

我使用了预训练的High Resolution，得到的ap值如下：

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] = 0.401
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets= 20 ] = 0.617
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets= 20 ] = 0.429
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.481
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.386
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] = 0.632
 Average Recall     (AR) @[ IoU=0.50      | area=   all | maxDets= 20 ] = 0.876
 Average Recall     (AR) @[ IoU=0.75      | area=   all | maxDets= 20 ] = 0.680
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.561
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.733

但README中给出的ap值是0.495，远高于测试的结果。请问预训练模型是不是传错了？

Answer 1 · 2019-11-29T17:34:42.000Z

好的收到，我double check 一下。

Answer 2 · 2019-11-29T20:09:27.000Z

Hi, @feipxyz I run the evaluation again to confirm the mAP is 0.495:

The command is simply python evaluate.py --cfg ../experiments/hrnet_w32_512.py....

Answer 3 · 2019-11-30T14:20:38.000Z

感谢回复。我想会不会是pytorch版本的问题。我是用pytorch1.0.1，因为使用pytorch0.4.1的时候，在编译DCNv2的时候会出现以下错误:

/data/Github/centerpose/lib/models/networks/DCNv2/src/cpu/vision.h:2:29: 致命错误：torch/extension.h：没有那个文件或目录

请问你使用的pytorch是哪个版本？

Answer 4 · 2019-11-30T18:36:14.000Z

同学你好，我的pytorch版本是1.1。

Answer 5 · 2019-12-02T02:53:25.000Z

同学你好，我的pytorch版本是1.1。

我使用python3.6-pytorch1.1-cuda9.2，编译DCNv2的时候有些告警，但是编译成功，但使用时出现未定义的符号错误，很奇怪我使用python3.6-pytorch1.0-cuda9.2的时候就没问题。错误如下：

Traceback (most recent call last):
  File "evaluate.py", line 22, in <module>
    from detectors.detector_factory import detector_factory
  File "/data/pengfei/Github/centerpose/tools/../lib/detectors/detector_factory.py", line 5, in <module>
    from .multi_pose import MultiPoseDetector
  File "/data/pengfei/Github/centerpose/tools/../lib/detectors/multi_pose.py", line 22, in <module>
    from .base_detector import BaseDetector
  File "/data/pengfei/Github/centerpose/tools/../lib/detectors/base_detector.py", line 12, in <module>
    from models.model import create_model, load_model
  File "/data/pengfei/Github/centerpose/tools/../lib/models/model.py", line 12, in <module>
    from .networks.pose_dla_dcn import get_pose_net as get_dla_dcn
  File "/data/pengfei/Github/centerpose/tools/../lib/models/networks/pose_dla_dcn.py", line 16, in <module>
    from .DCNv2.dcn_v2 import DCN
  File "/data/pengfei/Github/centerpose/tools/../lib/models/networks/DCNv2/dcn_v2.py", line 13, in <module>
    import _ext as _backend
ImportError: /data/pengfei/Github/centerpose/lib/models/networks/DCNv2/_ext.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC1ENS_14SourceLocationERKSs

编译DCNv2的过程如下：

running build
running build_ext
building '_ext' extension
creating build
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/data
creating build/temp.linux-x86_64-3.6/data/pengfei
creating build/temp.linux-x86_64-3.6/data/pengfei/Github
creating build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose
creating build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib
creating build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models
creating build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks
creating build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2
creating build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src
creating build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cpu
creating build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda
gcc -pthread -B /data/pengfei/anaconda3/envs/pytorch1.1/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/TH -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/data/pengfei/anaconda3/envs/pytorch1.1/include/python3.6m -c /data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/vision.cpp -o build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/vision.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++11
cc1plus: 警告：command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
gcc -pthread -B /data/pengfei/anaconda3/envs/pytorch1.1/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/TH -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/data/pengfei/anaconda3/envs/pytorch1.1/include/python3.6m -c /data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cpu/dcn_v2_cpu.cpp -o build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cpu/dcn_v2_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++11
cc1plus: 警告：command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/usr/local/cuda/bin/nvcc -DWITH_CUDA -I/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/TH -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/data/pengfei/anaconda3/envs/pytorch1.1/include/python3.6m -c /data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_cuda.cu -o build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++11
/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(83): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("lowest") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(84): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("max") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(85): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("lower_bound") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(86): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("upper_bound") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/c10/util/ArrayRef.h:277:48: 警告：‘deprecated’属性指定被忽略 [-Wattributes]
 using IntList C10_DEPRECATED_USING = ArrayRef<int64_t>;
                                                ^
/usr/local/cuda/bin/nvcc -DWITH_CUDA -I/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/TH -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/data/pengfei/anaconda3/envs/pytorch1.1/include/python3.6m -c /data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_im2col_cuda.cu -o build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_im2col_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++11
/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(83): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("lowest") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(84): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("max") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(85): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("lower_bound") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(86): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("upper_bound") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/c10/util/ArrayRef.h:277:48: 警告：‘deprecated’属性指定被忽略 [-Wattributes]
 using IntList C10_DEPRECATED_USING = ArrayRef<int64_t>;
                                                ^
/usr/local/cuda/bin/nvcc -DWITH_CUDA -I/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/TH -I/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/data/pengfei/anaconda3/envs/pytorch1.1/include/python3.6m -c /data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_psroi_pooling_cuda.cu -o build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_psroi_pooling_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++11
/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(83): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("lowest") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(84): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("max") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(85): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("lower_bound") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/cuda/NumericLimits.cuh(86): warning: calling a constexpr __host__ function("from_bits") from a __host__ __device__ function("upper_bound") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/c10/util/ArrayRef.h:277:48: 警告：‘deprecated’属性指定被忽略 [-Wattributes]
 using IntList C10_DEPRECATED_USING = ArrayRef<int64_t>;
                                                ^
/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_psroi_pooling_cuda.cu: 在 lambda 函数中:
/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_psroi_pooling_cuda.cu:317:120: 警告：‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated (declared at /data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/Dispatch.h:47) [-Wdeprecated-declarations]
   AT_DISPATCH_FLOATING_TYPES(input.type(), "dcn_v2_psroi_pooling_cuda_forward", [&] {
                                                                                                                        ^
/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_psroi_pooling_cuda.cu: 在 lambda 函数中:
/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_psroi_pooling_cuda.cu:391:126: 警告：‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated (declared at /data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/include/ATen/Dispatch.h:47) [-Wdeprecated-declarations]
   AT_DISPATCH_FLOATING_TYPES(out_grad.type(), "dcn_v2_psroi_pooling_cuda_backward", [&] {
                                                                                                                              ^
creating build/lib.linux-x86_64-3.6
g++ -pthread -shared -B /data/pengfei/anaconda3/envs/pytorch1.1/compiler_compat -L/data/pengfei/anaconda3/envs/pytorch1.1/lib -Wl,-rpath=/data/pengfei/anaconda3/envs/pytorch1.1/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/vision.o build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cpu/dcn_v2_cpu.o build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_cuda.o build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_im2col_cuda.o build/temp.linux-x86_64-3.6/data/pengfei/Github/centerpose/lib/models/networks/DCNv2/src/cuda/dcn_v2_psroi_pooling_cuda.o -L/usr/local/cuda/lib64 -lcudart -o build/lib.linux-x86_64-3.6/_ext.cpython-36m-x86_64-linux-gnu.so
running develop
running egg_info
creating DCNv2.egg-info
writing DCNv2.egg-info/PKG-INFO
writing dependency_links to DCNv2.egg-info/dependency_links.txt
writing top-level names to DCNv2.egg-info/top_level.txt
writing manifest file 'DCNv2.egg-info/SOURCES.txt'
reading manifest file 'DCNv2.egg-info/SOURCES.txt'
writing manifest file 'DCNv2.egg-info/SOURCES.txt'
running build_ext
copying build/lib.linux-x86_64-3.6/_ext.cpython-36m-x86_64-linux-gnu.so ->
Creating /data/pengfei/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/DCNv2.egg-link (link to .)
DCNv2 0.1 is already the active version in easy-install.pth

Installed /data/pengfei/Github/centerpose/lib/models/networks/DCNv2
Processing dependencies for DCNv2==0.1
Finished processing dependencies for DCNv2==0.1

Answer 6 · 2019-12-02T02:57:09.000Z

可以把 DCNv2文件夹下面的 build, DCNv2.egg-info 和 *.so文件删了重新编译下看看。

Answer 7 · 2019-12-02T03:10:39.000Z

可以把 DCNv2文件夹下面的 build, DCNv2.egg-info 和 *.so文件删了重新编译下看看。

我试过了也是不行，之前在使用CenterNet的时候也出现这种情况，当时换成pytorch0.4.1就可以了。
你使用的cuda是哪个版本，我换成相同版本的cuda再试试

Answer 8 · 2019-12-02T03:13:03.000Z

恩我用的是10.1，先试试这个。

Answer 9 · 2019-12-03T03:40:28.000Z

恩我用的是10.1，先试试这个。

结果有差别很可能是opencv版本的问题，opencv不同的版本可能底层使用的JPEG库版本不一样，读出来的图片数据有细微的差别。我这里使用的opencv是4.1版本，请问你使用的是哪个版本？

Answer 10 · 2019-12-20T19:54:36.000Z

Hi @feipxyz It's 4.1.1.26