data input question tf.keras.utils.Sequence -> tf.data
victusfate opened this issue · 0 comments
victusfate commented
I've been working on a fork of a fork (with an http server) locally and I'm seeing some warning messages
WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.
After reading up a bit and browsing around I came across this suggestion for how to migrate sequences w/ multiprocessing to tf.data generators to avoid deadlocks.
https://stackoverflow.com/a/63018990/51700
Are there any plans to move to new data input formats?