dpressel/rude-carnie

Cannot work on GPU and ask for Requirements

ziweiji opened this issue · 0 comments

I occurred an error when I run guess.py with GPU, but it's ok to run guess.py with CPU.

CUDA_VISIBLE_DEVICES=1 python guess.py --class_type gender --model_type inception \
--model_dir models/21936/ \
--face_detection_model models/shape_predictor_68_face_landmarks.dat --face_detection_type dlib \
--single_look True --device_id /device:gpu:0 \
--folder screenshot --out_dir results_try

The error is

2 root error(s) found.
(0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[node InceptionV3/InceptionV3/Conv2d_1a_3x3/Conv2D (defined at /home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:1748) ]]
(1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[node InceptionV3/InceptionV3/Conv2d_1a_3x3/Conv2D (defined at /home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:1748) ]]
[[Softmax/_761]]

Original stack trace for 'InceptionV3/InceptionV3/Conv2d_1a_3x3/Conv2D':
File "guess.py", line 246, in
tf.app.run()
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/absl/app.py", line 303, in run
_run_main(main, args)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "guess.py", line 179, in main
logits = model_fn(nlabels, images, 1, False)
File "/home/jiziwei/Movie/rude-carnie/model.py", line 89, in inception_v3
net, end_points = inception_v3_base(images, scope=scope)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/contrib/slim/python/slim/nets/inception_v3.py", line 112, in inception_v3_base
net = layers.conv2d(inputs, depth(32), [3, 3], stride=2, scope=end_point)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/contrib/framework/python/ops/arg_scope.py", line 182, in func_with_args
return func(*args, **current_args)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py", line 1159, in convolution2d
conv_dims=2)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/contrib/framework/python/ops/arg_scope.py", line 182, in func_with_args
return func(*args, **current_args)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py", line 1057, in convolution
outputs = layer.apply(inputs)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 1700, in apply
return self.call(inputs, *args, **kwargs)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/layers/base.py", line 548, in call
outputs = super(Layer, self).call(inputs, *args, **kwargs)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 854, in call
outputs = call_fn(cast_inputs, *args, **kwargs)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/autograph/impl/api.py", line 234, in wrapper
return converted_call(f, options, args, kwargs)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/autograph/impl/api.py", line 439, in converted_call
return _call_unconverted(f, args, kwargs, options)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/autograph/impl/api.py", line 330, in _call_unconverted
return f(*args, **kwargs)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/keras/layers/convolutional.py", line 197, in call
outputs = self._convolution_op(inputs, self.kernel)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/ops/nn_ops.py", line 1134, in call
return self.conv_op(inp, filter)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/ops/nn_ops.py", line 639, in call
return self.call(inp, filter)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/ops/nn_ops.py", line 238, in call
name=self.name)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/ops/nn_ops.py", line 2010, in conv2d
name=name)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_nn_ops.py", line 1071, in conv2d
data_format=data_format, dilations=dilations, name=name)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/home/jiziwei/anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()

I think the problem is about the version of cuDNN and TF.
My environment is:

cuDNN version: 4.7.1
Cuda version: 10.0
TensorFlow version: 1.15.0

Could you please offer the Requirements for the version of packages?