Error while Execution
Closed this issue · 6 comments
ERROR train_luong_tf2.py
Shows this error while execution. How do i fix it?
File "train_luong_tf2.py", line 331, in
target_seq_out, en_initial_states)
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 426, in call
self.initialize(args, kwds, add_initializers_to=initializer_map)
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 370, in initialize
*args, **kwds))
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1313, in get_concrete_function_internal_garbage_collected
graph_function, , _ = self.maybe_define_function(args, kwargs)
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1580, in maybe_define_function
graph_function = self.create_graph_function(args, kwargs)
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1512, in create_graph_function
capture_by_value=self.capture_by_value),
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 694, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 317, in wrapped_fn
return weak_wrapped_fn().wrapped(*args, **kwds)
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 686, in wrapper
), args, kwargs)
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py", line 392, in converted_call
result = converted_f(*effective_args, **kwargs)
File "/tmp/tmpxzo8k595.py", line 19, in tf__train_step
gradients = ag.converted_call('gradient', tape, ag.ConversionOptions(recursive=True, verbose=0, strip_decorators=(tf.function, defun_5, ag.convert, ag.do_not_convert, ag_.converted_call), force_conversion=False, optional_features=(), internal_convert_user_code=True), (loss, variables), {})
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py", line 267, in converted_call
return _call_unconverted(f, args, kwargs)
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py", line 188, in _call_unconverted
return f(*args, **kwargs)
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/eager/backprop.py", line 956, in gradient
unconnected_gradients=unconnected_gradients)
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/eager/imperative_grad.py", line 72, in imperative_grad
compat.as_str(unconnected_gradients.value))
File "/home/itachi/venv/lib/python3.6/site-packages/tensorflow/python/eager/backprop.py", line 554, in _aggregate_grads
if len(gradients) == 1:
SystemError: returned a result with an error set
Hmm, seems like I cannot reproduce this at my local machine.
Did you run from Tensorflow 2.0's container?
Hi, yeah am using tensorflow '2.0.0-alpha0'. Also there is a missing import requests in the code.
Sorry, my bad, it's not what I meant. You installed Tensorflow 2.0 directly on your machine instead of using Docker, didn't you?
Yeah, i did. Do i need to do it using Docker?
Since I used Docker for all my experiments, I recommend that you run it from Tensorflow 2.0 container first. It's unclear that which is causing the problem, can be either my code or your environment.
SystemError: returned a result with an error set
I searched that error on Google, seems like it has something to do with some Python dependency.
Anyway, give Docker a try and tell me whether that error still occurs. I would appreciate that.
Temporarily close this issue. Reopen if the error still occurs.