alfg/ffprobe-wasm

Matroska file support

Banou26 opened this issue · 11 comments

Banou26 commented

Would it be possible to support .mkv files ?
Currently when trying to probe a matroska file, it errors out with RuntimeError: table index is out of bounds

alfg commented

Hi @Banou26,

I have updated the ffmpeg wasm build to include the matroska demuxer and updated the website: https://alfg.github.io/ffprobe-wasm/ It should now work.

I currently have the demuxer and decoder support limited to keep the wasm file size small. However, I plan to find an optimal build size that contains most common demuxer and decoder support. It is now set to the following:

  --enable-decoder=h264,aac,pcm_s16le \
  --enable-demuxer=mov,matroska \

Thanks!

  • 👍1
  • ❤️1
Banou26 commented

Hey, thanks for this lib btw :)
There is just two problems I encountered while trying to use this lib with matroska files:

  • When using window.Module.get_frames(0), the nb_frames is always 0
  • I get a OOM error that uses the thread at 100% after using window.Module.get_frames(pageNumber) like 4-5 times
alfg commented

Hi @Banou26, thanks for the heads up. I'll take a look into the frames issue soon.

  • ❤️1
Banou26 commented

So the hanging problem seem to be because webworkers are spawned when using window.Module.get_frames(0) but never closed, and after 8 webworkers, the next call errors out with oom, and the next call result in a 100% cpu usage because it tries to run it in the main thread & just hangs up blocking forever

alfg commented

Hey, sorry I haven't had a chance to work on this yet. But yeah, it sounds like get_frames could use some work to close or cleanup the paging.

alfg commented

Hi @Banou26,

I added a fix for the OOM issue and nb_frames for MKV.

For some context:

  1. Similar to WebM, nb_frames is reported as 0 for MKV from libav's AVFormatContext->streams[i].nb_frames. So instead, I calculate the frames via duration and fps. It is similar to this issue: https://stackoverflow.com/questions/32532122/finding-duration-number-of-frames-of-webm-using-ffmpeg-libavformat

  2. I forgot to cleanup the allocations when iterating the frames data, so it was building up leading to the OOM issue. Hopefully the performance is a bit better too.

I have published the latest to https://alfg.github.io/ffprobe-wasm/. Let me know if this works for you.

Thanks!

  • 👍1
  • ❤️1
Banou26 commented

Hey, I happened to fall on multiple MKV files that have very wrong frame pages.

For example, to get new frames I have to go from pages (48 * x), and x is 1 -> 2 -> 4 -> 6 -> 8.
The page 3 is equal to 2, 5 is equal to 4, ect...
And their PTS & DTS differs so much from page to page that it looks like we miss an entire page of frames(which would coincide with the duplicate odd page).

Also I found some other files that had pages with a weird ordering.
e.g, page 4 has DTS from 7000 to 9000, and page 5 with DTS from 8000 to 10000.

If you could take a look at this problem, I'd appreciate it.
If you can't manage to find MKV files that have this problem, I could probably give you a sample, but it's copyrighted material so I'm not sure how to share it.

  • 👀1
alfg commented

Ok, will take a look into this next after fixing the main thread blocking issue.

  • 👍1
alfg commented

OK. I think this has to do with the paginated size (48) vs the GOP size of the video. I have it set to process 48 frames at a time, assuming a fixed GOP size of 48. However, if your GOP size differs, then it will process starting at the next keyframe since it requires the keyframe to decode the rest. However, the pagination won't work for variable GOP, since I can't predetermine the total frames size.

I'm adding a fix soon to find the GOP size and use that as the perPage paginated results and hopefully that will fix it for fixed GOP.

I know there's more work to do around more filtering on the paginated results since it can get tedious when working with a lot of frames. I might change this to time based. Honestly, this was an experimental project to get Wasm working with libav*. 😄 Hopefully I'll have some time to work on this functionality too.

Edit: I published the latest to https://alfg.github.io/ffprobe-wasm/ if you want to try it with your MKV file.

  • 👍1
  • ❤️1
alfg commented

Updated again with a timestamp slider input.

alfg commented

Closing for now. Feel free to re-open or create a new issue if there's anything else. Thanks!

  • 👍1