xv44586/ccf_2020_qa_match

自蒸馏脚本运行失败

ljch2018 opened this issue · 1 comments

自蒸馏脚本错误日志如下:

Traceback (most recent call last):
  File "pair-self-kd.py", line 297, in <module>
    callbacks=[student_evaluator])
  File "/home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/keras/engine/training.py", line 1732, in fit_generator
    initial_epoch=initial_epoch)
  File "/home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/keras/engine/training_generator.py", line 220, in fit_generator
    reset_metrics=False)
  File "/home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/keras/engine/training.py", line 1514, in train_on_batch
    outputs = self.train_function(ins)
  File "/home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/tensorflow/python/keras/backend.py", line 3792, in __call__
    outputs = self._graph_fn(*converted_inputs)
  File "/home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1605, in __call__
    return self._call_impl(args, kwargs)
  File "/home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1645, in _call_impl
    return self._call_flat(args, self.captured_inputs, cancellation_manager)
  File "/home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1746, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 598, in call
    ctx=ctx)
  File "/home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.FailedPreconditionError: 2 root error(s) found.
  (0) Failed precondition:  Error while reading resource variable _AnonymousVar409 from Container: localhost. This could mean that the variable was uninitialized. Not found: Resource localhost/_AnonymousVar409/N10tensorflow3VarE does not exist.
         [[node ReadVariableOp_1191 (defined at /home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3009) ]]
         [[ReadVariableOp_1190/_12]]
  (1) Failed precondition:  Error while reading resource variable _AnonymousVar409 from Container: localhost. This could mean that the variable was uninitialized. Not found: Resource localhost/_AnonymousVar409/N10tensorflow3VarE does not exist.
         [[node ReadVariableOp_1191 (defined at /home/work/.conda/envs/py3-tf.2.2-ccf/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3009) ]]
0 successful operations.
0 derived errors ignored. [Op:__inference_keras_scratch_graph_150548]

Function call stack:
keras_scratch_graph -> keras_scratch_graph

看上去是你的环境问题,按requirements.txt 对齐一下环境吧,尤其是keras和tf