Using the rooms data?
Closed this issue · 3 comments
Amir-Arsalan commented
I wanted to try your code on the rooms data but during conversion, I get these errors. What could I be doing wrong?
Note that for the rooms data with moving camera I set the number of camera parameters to 7:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "tfrecord-converter.py", line 66, in convert
for i, batch in enumerate(batch_process(record)):
File "tfrecord-converter.py", line 29, in chunk
for first in iterator:
File "tfrecord-converter.py", line 40, in process
'cameras': tf.FixedLenFeature(shape=SEQ_DIM * POSE_DIM, dtype=tf.float32)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/parsing_ops.py", line 1019, in parse_single_example
serialized, features, example_names, name
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/parsing_ops.py", line 1063, in parse_single_example_v2_unoptimized
return parse_single_example_v2(serialized, features, name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/parsing_ops.py", line 2089, in parse_single_example_v2
dense_defaults, dense_shapes, name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/parsing_ops.py", line 2206, in _parse_single_example_v2_raw
name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_parsing_ops.py", line 1164, in parse_single_example
ctx=_ctx)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_parsing_ops.py", line 1260, in parse_single_example_eager_fallback
attrs=_attrs, ctx=_ctx, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py", line 67, in quick_execute
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: Key: frames. Can't parse serialized Example. [Op:ParseSingleExample]
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "tfrecord-converter.py", line 98, in <module>
pool.map(f, records)
File "/usr/lib/python3.6/multiprocessing/pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
tensorflow.python.framework.errors_impl.InvalidArgumentError: Key: frames. Can't parse serialized Example. [Op:ParseSingleExample]
wohlert commented
I can see that the rooms data has a different sequence length (10). The length is hardcoded here:
By changing it, you might have success.
Amir-Arsalan commented
@wohlert I tried what you said and set SEQ_DIM to 10 but got these error messages. Do you think you know if there is another part of your code that you think I should change your code so that I can get the conversion working for the rooms_ring_camera and rooms_free_camera_no_object_rotations?
2020-10-24 04:35:07.522434: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: cameras. Can't parse serialized Example.
2020-10-24 04:35:07.522598: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:198] kernel reported version is: 440.82.0
2020-10-24 04:35:07.522691: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:198] kernel reported version is: 440.82.0
2020-10-24 04:35:07.522840: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:198] kernel reported version is: 440.82.0
2020-10-24 04:35:07.522837: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:198] kernel reported version is: 440.82.0
2020-10-24 04:35:07.522987: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:198] kernel reported version is: 440.82.0
2020-10-24 04:35:07.523026: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:198] kernel reported version is: 440.82.0
2020-10-24 04:35:07.523760: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: frames. Can't parse serialized Example.
2020-10-24 04:35:07.525512: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: cameras. Can't parse serialized Example.
2020-10-24 04:35:07.525680: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: frames. Can't parse serialized Example.
2020-10-24 04:35:07.526211: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: cameras. Can't parse serialized Example.
2020-10-24 04:35:07.526335: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: cameras. Can't parse serialized Example.
2020-10-24 04:35:07.526344: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: frames. Can't parse serialized Example.
2020-10-24 04:35:07.526377: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: frames. Can't parse serialized Example.
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "tfrecord-converter.py", line 67, in convert
for i, batch in enumerate(batch_process(record)):
File "tfrecord-converter.py", line 30, in chunk
for first in iterator:
File "tfrecord-converter.py", line 41, in process
'cameras': tf.FixedLenFeature(shape=SEQ_DIM * POSE_DIM, dtype=tf.float32)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/parsing_ops.py", line 800, in parse_single_example
return parse_single_example_v2(serialized, features, name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/parsing_ops.py", line 1688, in parse_single_example_v2
dense_defaults, dense_shapes, name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/parsing_ops.py", line 1805, in _parse_single_example_v2_raw
name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_parsing_ops.py", line 1040, in parse_single_example
ctx=_ctx)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_parsing_ops.py", line 1084, in parse_single_example_eager_fallback
attrs=_attrs, ctx=_ctx, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py", line 66, in quick_execute
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: Key: frames. Can't parse serialized Example. [Op:ParseSingleExample]
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "tfrecord-converter.py", line 99, in <module>
pool.map(f, records)
File "/usr/lib/python3.6/multiprocessing/pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
tensorflow.python.framework.errors_impl.InvalidArgumentError: Key: frames. Can't parse serialized Example. [Op:ParseSingleExample]
wohlert commented
Try out this one https://github.com/l3robot/gqn_datasets_translator and see if it can help you. My script is based on this, with the one I provided adding multiprocessing. It could be that I have introduced some bugs.