OutOfRangeError: I am getting this Error while training.
VRathi03 opened this issue · 1 comments
Data Loading...
Starting Train1
WARNING:tensorflow:From /disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/input.py:187: QueueRunner.init (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the tf.data module.
WARNING:tensorflow:From /disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/input.py:187: add_queue_runner (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the tf.data module.
2018-10-22 17:15:41.600461: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
Inside TF Session
WARNING:tensorflow:From train1.py:60: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the tf.data module.
Inside Epoch
0%| | 0/144 [00:00<?, ?b/s]Inside For loop
Inside Queue
2018-10-22 17:15:55.014412: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015069: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015102: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015774: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015814: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015822: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015828: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015835: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015841: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015862: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015920: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015940: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015947: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015952: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015958: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015968: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015976: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015985: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.015992: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016004: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016013: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016024: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016033: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016044: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016059: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016069: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016081: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016091: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016102: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016112: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016122: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-10-22 17:15:55.016132: W tensorflow/core/kernels/queue_base.cc:285] _1_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
Traceback (most recent call last):
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1292, in _do_call
return fn(*args)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1277, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1367, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.OutOfRangeError: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 32, current size 0)
[[{{node batch}} = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/padding_fifo_queue, batch/n)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train1.py", line 106, in
train(logdir=logdir)
File "train1.py", line 68, in train
sess.run(train_op)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 887, in run
run_metadata_ptr)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1110, in _run
feed_dict_tensor, options, run_metadata)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1286, in _do_run
run_metadata)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1308, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 32, current size 0)
[[{{node batch}} = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/padding_fifo_queue, batch/n)]]
Caused by op 'batch', defined at:
File "train1.py", line 106, in
train(logdir=logdir)
File "train1.py", line 19, in train
model = Model(mode="train1", batch_size=hp.Train1.batch_size, queue=queue)
File "/disk2/WorkSpace/Voice_Transformation/voice_transformation_staging/models.py", line 22, in init
self.x_mfcc, self.y_ppgs, self.y_spec, self.y_mel, self.num_batch = self.get_input(mode, batch_size, queue)
File "/disk2/WorkSpace/Voice_Transformation/voice_transformation_staging/models.py", line 51, in get_input
x_mfcc, y_ppgs, num_batch = get_batch_queue(mode=mode, batch_size=batch_size)
File "/disk2/WorkSpace/Voice_Transformation/voice_transformation_staging/data_load.py", line 224, in get_batch_queue
dynamic_pad=True)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/input.py", line 989, in batch
name=name)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/input.py", line 763, in _batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/data_flow_ops.py", line 476, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3481, in queue_dequeue_many_v2
component_types=component_types, timeout_ms=timeout_ms, name=name)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3272, in create_op
op_def=op_def)
File "/disk2/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1768, in init
self._traceback = tf_stack.extract_stack()
OutOfRangeError (see above for traceback): PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 32, current size 0)
[[{{node batch}} = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/padding_fifo_queue, batch/n)]]
I have a same problem. Did you solve this problem ?