ERROR when doing training
YongyuG opened this issue · 1 comments
HI,
thanks for you works
I tried to implement your codes in python 3.7 and latest tensorflow
There is a bug confused me a lot
Can you tell me how to fix it
`[*] Reading checkpoints...
[!] Load failed
2019-06-11 14:48:50.084825: E tensorflow/core/framework/op_segment.cc:54] Create kernel failed: Not found: No registered 'TemporaryVariable' OpKernel for XLA_CPU devices compatible with node {{node device_1/gradients/AddN_18/tmp_var}}
. Registered: device='CPU'
2019-06-11 14:48:50.084855: E tensorflow/core/common_runtime/executor.cc:624] Executor failed to create kernel. Not found: No registered 'TemporaryVariable' OpKernel for XLA_CPU devices compatible with node {{node device_1/gradients/AddN_18/tmp_var}}
. Registered: device='CPU'
[[{{node device_1/gradients/AddN_18/tmp_var}}]]
2019-06-11 14:48:50.105205: W tensorflow/core/kernels/queue_base.cc:277] _0_device_0/input_producer: Skipping cancelled enqueue attempt with queue not closed
2019-06-11 14:48:50.105333: W tensorflow/core/kernels/queue_base.cc:277] _3_device_1/wav_and_noisy/random_shuffle_queue: Skipping cancelled enqueue attempt with queue not closed
2019-06-11 14:48:50.105410: W tensorflow/core/kernels/queue_base.cc:277] _3_device_1/wav_and_noisy/random_shuffle_queue: Skipping cancelled enqueue attempt with queue not closed
2019-06-11 14:48:50.105546: W tensorflow/core/kernels/queue_base.cc:277] _1_device_0/wav_and_noisy/random_shuffle_queue: Skipping cancelled enqueue attempt with queue not closed
2019-06-11 14:48:50.105554: W tensorflow/core/kernels/queue_base.cc:277] _1_device_0/wav_and_noisy/random_shuffle_queue: Skipping cancelled enqueue attempt with queue not closed
Traceback (most recent call last):
File "/home/yongyug/anaconda3/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/home/yongyug/anaconda3/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/yongyug/anaconda3/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.NotFoundError: No registered 'TemporaryVariable' OpKernel for XLA_CPU devices compatible with node {{node device_1/gradients/AddN_18/tmp_var}}
. Registered: device='CPU'
[[{{node device_1/gradients/AddN_18/tmp_var}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 104, in
tf.app.run()
File "/home/yongyug/anaconda3/lib/python3.7/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "main.py", line 82, in main
se_model.train(FLAGS, udevices)
File "/home/yongyug/Gao/segan/model.py", line 399, in train
self.d_rl_losses[0]])
File "/home/yongyug/anaconda3/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 929, in run
run_metadata_ptr)
File "/home/yongyug/anaconda3/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "/home/yongyug/anaconda3/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
run_metadata)
File "/home/yongyug/anaconda3/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: No registered 'TemporaryVariable' OpKernel for XLA_CPU devices compatible with node {{node device_1/gradients/AddN_18/tmp_var}}
. Registered: device='CPU'
[[{{node device_1/gradients/AddN_18/tmp_var}}]]
`
Faced same issue. I tried reducing batch size and it worked