tensorflow.python.framework.errors_impl.DataLossError: corrupted record at 2585605691
Closed this issue · 1 comments
This issue came out after 50000 steps during the training phase. Something like this,
restoring checkpoint /2D_projection_matching-main/checkpoints/lr-0.0001_dataset-02958343_pointn-1024_gtp-5000/32/model-50000
INFO:tensorflow:Restoring parameters from /2D_projection_matching-main/checkpoints/lr-0.0001_dataset-02958343_pointn-1024_gtp-5000/32/model-50000
File "/2D_projection_matching-main/2Dpm/main/../util/simple_dataset.py", line 38, in load_data
for k, string_record in enumerate(record_iterator):
File "/python3.6/site-packages/tensorflow/python/lib/io/tf_record.py", line 174, in tf_record_iterator
reader.GetNext()
File "/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 490, in GetNext
return _pywrap_tensorflow_internal.PyRecordReader_GetNext(self)
tensorflow.python.framework.errors_impl.DataLossError: corrupted record at 2585605691
Could you please do me a favor on solving this issue? Many thanks.