训练报错
hjgherman opened this issue · 5 comments
请问训练是报这个错什么原因
Traceback (most recent call last):
File "", line 1, in
File "D:\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "D:\Anaconda3\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
@hjgherman 您好，麻烦发完整的错误信息，不清楚是哪里抛出的异常
你好，我训练也报错，您看一下是什么原因
WARNING:tensorflow:From C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\site-packages\tensorflow\python\ops\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\site-packages\tensorflow\python\ops\gradients_impl.py:110: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
Epoch 1/80
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\site-packages\keras\utils\data_utils.py", line 666, in _run
with closing(self.executor_fn(_SHARED_SEQUENCES)) as executor:
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\site-packages\keras\utils\data_utils.py", line 661, in
initargs=(seqs, self.random_seed))
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\pool.py", line 174, in init
self._repopulate_pool()
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\pool.py", line 239, in _repopulate_pool
w.start()
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle generator objects
Exception in thread Thread-3:
Traceback (most recent call last):
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\site-packages\keras\utils\data_utils.py", line 666, in _run
with closing(self.executor_fn(_SHARED_SEQUENCES)) as executor:
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\site-packages\keras\utils\data_utils.py", line 661, in
initargs=(seqs, self.random_seed))
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\pool.py", line 174, in init
self._repopulate_pool()
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\pool.py", line 239, in _repopulate_pool
w.start()
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle generator objects
Using TensorFlow backend.
Using TensorFlow backend.
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "C:\Users\AIDONG\AppData\Local\conda\conda\envs\py3_lenet\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
@Scheaven 不清楚啥原因，也许是版本问题？以下是我的环境主要包版本，python 是3.6
scikit-image 0.13.0
scikit-learn 0.20.3
scipy 1.1.0
Keras 2.2.4
numpy 1.15.0
tensorboard 1.9.0
tensorflow-gpu 1.9.0
谢谢，我的问题避免了，我是在6G显存上的GTX1060上训练出现的问题，但是我换成了8G显存的GTX1080就没有出现这个问题了。但是具体原因我也不清点。