Error: Can't pickle generator

Question

Error: Can't pickle generator

yogeshhk opened this issue 4 years ago · 7 comments

On Windows 10, with Python 3.6.5, getting error on running

tf2_gnn_train RGCN PPI --max-epochs 10 data/ppi/

log is:

2021-06-10 18:44:04.837559: W tensorflow/core/framework/op_kernel.cc:1755] Invalid argument: TypeError: cannot pickle 'generator' object
Traceback (most recent call last):

  File "C:\Users\yogesh.kulkarni\AppData\Local\Continuum\anaconda3\envs\tf2\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 789, in get_iterator
    return self._iterators[iterator_id]

KeyError: 0

Any resolution?

Answer 1 · 2021-06-10T15:22:47.000Z

Could you tell what Tensorflow version you were using?

Answer 2 · 2021-06-11T00:13:21.000Z

tensorflow 2.5
Python 3.8.8
CPU

Answer 3 · 2021-08-01T15:04:15.000Z

I faced the same problem, how to fix it bro?

Answer 4 · 2021-08-02T01:47:51.000Z

@lh98cool: did not find any solution yet.

Answer 5 · 2021-12-02T16:28:16.000Z

Any updates?

Answer 6 · 2021-12-20T10:19:11.000Z

I have found a temporary fix for this issue.
In file graph_dataset.py, line 294, make a list out of self.graph_batch_iterator(data_fold)
E.g. it should look like something like this:
if use_worker_threads: graph_batch_iterator = lambda: DoubleBufferedIterator( list(self.graph_batch_iterator(data_fold)) )

Alternatively, disable multithreding by setting use_worker_threads to False in line 79.

Answer 7 · 2022-07-05T11:42:32.000Z

I got the same error on mac(Big Sur), but it worked on Colab so for now I'm using that.