commaai/comma2k19

openpilot_tools.lib.exceptions.DataUnreadableError

Jiankai-Sun opened this issue · 8 comments

Jiankai-Sun commented

There is a openpilot_tools.lib.exceptions.DataUnreadableError when I use openpilot_tools to read Chunk_1/b0c9d2329ad1606b|2018-07-29--16-37-17/5/video.hevc . There are two possible reasons:

  • The installation of openpilot_tools is uncompleted.
  • The datasets are broken, e.g. Chunk_1/b0c9d2329ad1606b|2018-07-29--16-37-17/5/video.hevc

I believe that I have installed openpilot_tools successfully. Do you have any suggestions? Thank you!

Error details:

(mapnet_release) [sun@pc]$ python
Python 2.7.15 |Anaconda, Inc.| (default, Oct 10 2018, 21:32:13) 
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from openpilot_tools.lib.framereader import FrameReader
>>> fr = FrameReader('video.hevc')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 869, in FrameReader
    index_data = get_video_index(fn, frame_type, cache_prefix)
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 201, in get_video_index
    index_video(fn, frame_type, cache_prefix)
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 193, in index_video
    index_stream(fn, "hevc", cache_prefix=cache_prefix)
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 119, in cache_inner
    cache_value = func(fn, *args, **kwargs)
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 136, in index_stream
    probe = ffprobe(f.name, typ)
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 79, in ffprobe
    raise DataUnreadableError(fn)
openpilot_tools.lib.exceptions.DataUnreadableError: video.hevc
adhintz commented

At the start of your python session, also run this command:
import os

Jiankai-Sun commented

At the start of your python session, also run this command:
import os

How did you find out this trick? Or Why does it work?

adhintz commented

Your original error message had

>>> fr = FrameReader(os.path.join('video.hevc'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'os' is not defined

which indicates that os.path.join() did not work because python could not find the name os. os is a common module in python, but it needs to be imported before you can use it.

  • ❤️1
Jiankai-Sun commented

Oh, I have updated the issue. That's not the real reason. Thank you all the same.
I am afraid the dataset is broken. Have you successfully loaded the data using the provided API?

haraschax commented

I installed the tools on a new device and downloaded that chunk and it does work for me. There must be a different issue. Can you successfully run the example notebooks provided on the data provided in this repo?

Jiankai-Sun commented

@HaraldSchafer Yes, I cansuccessfully run the example notebooks provided on the data provided in this repo.

FrancoisMasson1990 commented

@Jiankai-Sun I had the same error. You just need to remove the if condition in ffprobe(fn, fmt=None) in framereader.py
It correspond to the lignes 72-73 in the python file.

  • 👍2
gucasbrg commented

this cmd in order to get video info,
if you can use "ffprobe -v quiet -print_format json -show_format -show_streams -format hevc /data/openpilot-data/b0c9d2329ad1606b|2018-09-21--23-18-39/0/video.hevc
"
Recompile ffmpeg without any output

  • 👍3