[Bug]: frame extraction by timestamp operation can be very slow
Opened this issue · 1 comments
Description
This recipe demonstrated extracting a single frame by a timestamp via the FFdecoder API.
However, the frame extraction time goes up when the timestamp gets bigger.
The extraction time is so slow such that this API is not usable for my use case. Thus, filing a bug report for an expedited fix.
Looked deeper into the generated ffmpeg command, I found that the generated command is not optimal. A better approach is to use Input Seeking instead of Output Seeking, ref.
Here is an example generated command:
ffmpeg -vcodec hevc -i /Users/asia/Downloads/frames/HacksawRidge.2016.BluRay.1080p.x265.10bit.2Audio-MiniHD.mkv -pix_fmt rgb24 -s 1920x804 -framerate 23.98 -ss 00:30:00.00 -t 120s -r 0.5 -f rawvideo -
Notice that the -ss
flag is placed after the input file, i.e. -i video_file_path
.
If we place the -ss
flag before the input file, this operation becomes a lot more faster. E.g.
ffmpeg -vcodec hevc ss 00:30:00.00 -t 120s -i /Users/asia/Downloads/frames/HacksawRidge.2016.BluRay.1080p.x265.10bit.2Audio-MiniHD.mkv -pix_fmt rgb24 -s 1920x804 -framerate 23.98 -r 0.5 -f rawvideo -
Changing the ordering of the flag should be a quick fix for this issue.
Hoping we could get it fixed soon.
Issue Checklist
- I have searched open or closed issues for my problem and found nothing related or helpful.
- I have read the Documentation and found nothing related to my problem.
- I've read the Issue Guidelines and wholeheartedly agree.
Expected behaviour
Extracting a single frame by a timestamp via the FFdecoder API should be fast regardless of the timestamp.
Actual behaviour
The extraction time is so slow such that this API is not usable for my use case.
Steps to reproduce
from deffcode import FFdecoder
from PIL import Image
from pathlib import Path
video_path = "~/Downloads/frames/HacksawRidge.2016.BluRay.1080p.x265.10bit.2Audio-MiniHD.mkv"
ffparams = {
"-ss": "00:30:00.00",
"-t": "120s",
"-r": 2,
}
output = "output"
decoder = FFdecoder(video_path, verbose=True, **ffparams).formulate()
frame_count = 0
for frame in decoder.generateFrame():
if frame is None:
break
frame_count += 1
Image.fromarray(frame).save(f"{output}/frame_{frame_count}.jpeg")
# terminate the decoder
decoder.terminate()
Terminal log output
�[32m13:12:27�[0m :: �[1;35m FFhelper �[0m :: �[1;33m DEBUG �[0m :: �[1;37mFinal FFmpeg Path: ffmpeg�[0m
�[32m13:12:28�[0m :: �[1;35m FFhelper �[0m :: �[1;33m DEBUG �[0m :: �[1;37mFFmpeg validity Test Passed!�[0m
�[32m13:12:28�[0m :: �[1;35m FFhelper �[0m :: �[1;33m DEBUG �[0m :: �[1;37mFound valid FFmpeg Version: `b'6.0'` installed on this system�[0m
�[32m13:12:28�[0m :: �[1;35m Sourcer �[0m :: �[1;33m DEBUG �[0m :: �[1;37mFound valid FFmpeg executable: `ffmpeg`.�[0m
�[32m13:12:28�[0m :: �[1;35m Sourcer �[0m :: �[1;33m DEBUG �[0m :: �[1;37mExtracting Metadata...�[0m
�[32m13:12:28�[0m :: �[1;35m Sourcer �[0m :: �[1;33m DEBUG �[0m :: �[1;37mMetadata Extraction completed successfully!�[0m
�[32m13:12:28�[0m :: �[1;35m FFdecoder �[0m :: �[1;36m INFO �[0m :: �[1;37mUsing default `rgb24` pixel-format for this pipeline.�[0m
�[32m13:12:28�[0m :: �[1;35m FFdecoder �[0m :: �[1;36m INFO �[0m :: �[1;37mDefault source resolution will be used for this pipeline for defining output resolution.�[0m
�[32m13:12:28�[0m :: �[1;35m FFdecoder �[0m :: �[1;36m INFO �[0m :: �[1;37mDefault source framerate will be used for this pipeline for defining output framerate.�[0m
�[32m13:12:28�[0m :: �[1;35m FFdecoder �[0m :: �[1;31m�[47mCRITICAL�[0m :: �[1;37mActivating Video-Only Mode of Operation.�[0m
�[32m13:12:28�[0m :: �[1;35m FFdecoder �[0m :: �[1;33m DEBUG �[0m :: �[1;37mExecuting FFmpeg command: `ffmpeg -vcodec hevc -i /Users/asia/Downloads/frames/HacksawRidge.2016.BluRay.1080p.x265.10bit.2Audio-MiniHD.mkv -pix_fmt rgb24 -s 1920x804 -framerate 23.98 -ss 00:30:00.00 -t 120s -r 0.5 -f rawvideo -`�[0m
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
built with Apple clang version 14.0.0 (clang-1400.0.29.202)
configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-neon
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
[matroska,webm @ 0x148304130] Could not find codec parameters for stream 3 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 0x148304130] Could not find codec parameters for stream 4 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 0x148304130] Could not find codec parameters for stream 5 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 0x148304130] Could not find codec parameters for stream 6 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 0x148304130] Could not find codec parameters for stream 7 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, matroska,webm, from '/Users/asia/Downloads/frames/HacksawRidge.2016.BluRay.1080p.x265.10bit.2Audio-MiniHD.mkv':
Metadata:
encoder : libebml v1.3.4 + libmatroska v1.4.5
creation_time : 2017-02-13T00:44:49.000000Z
Duration: 02:19:19.49, start: 0.000000, bitrate: 6485 kb/s
Chapters:
Chapter #0:0: start 0.000000, end 558.765000
Metadata:
title : Chapter 01
Chapter #0:1: start 558.765000, end 1121.993000
Metadata:
title : Chapter 02
Chapter #0:2: start 1121.993000, end 1594.509000
Metadata:
title : Chapter 03
Chapter #0:3: start 1594.509000, end 1980.560000
Metadata:
title : Chapter 04
Chapter #0:4: start 1980.560000, end 2404.149000
Metadata:
title : Chapter 05
Chapter #0:5: start 2404.149000, end 3007.211000
Metadata:
title : Chapter 06
Chapter #0:6: start 3007.211000, end 3421.541000
Metadata:
title : Chapter 07
Chapter #0:7: start 3421.541000, end 3861.648000
Metadata:
title : Chapter 08
Chapter #0:8: start 3861.648000, end 4670.207000
Metadata:
title : Chapter 09
Chapter #0:9: start 4670.207000, end 5038.698000
Metadata:
title : Chapter 10
Chapter #0:10: start 5038.698000, end 5481.725000
Metadata:
title : Chapter 11
Chapter #0:11: start 5481.725000, end 5942.810000
Metadata:
title : Chapter 12
Chapter #0:12: start 5942.810000, end 6367.569000
Metadata:
title : Chapter 13
Chapter #0:13: start 6367.569000, end 7188.305000
Metadata:
title : Chapter 14
Chapter #0:14: start 7188.305000, end 7633.292000
Metadata:
title : Chapter 15
Chapter #0:15: start 7633.292000, end 8359.488000
Metadata:
title : Chapter 16
Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, unknown/bt709/unknown), 1920x804, SAR 1:1 DAR 160:67, 23.98 fps, 23.98 tbr, 1k tbn (default)
Metadata:
BPS : 5486317
BPS-eng : 5486317
DURATION : 02:19:19.435000000
DURATION-eng : 02:19:19.435000000
NUMBER_OF_FRAMES: 200426
NUMBER_OF_FRAMES-eng: 200426
NUMBER_OF_BYTES : 5732814506
NUMBER_OF_BYTES-eng: 5732814506
_STATISTICS_WRITING_APP: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_DATE_UTC: 2017-02-13 00:44:49
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-13 00:44:49
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
Metadata:
BPS : 640000
BPS-eng : 640000
DURATION : 02:19:19.424000000
DURATION-eng : 02:19:19.424000000
NUMBER_OF_FRAMES: 261232
NUMBER_OF_FRAMES-eng: 261232
NUMBER_OF_BYTES : 668753920
NUMBER_OF_BYTES-eng: 668753920
_STATISTICS_WRITING_APP: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_DATE_UTC: 2017-02-13 00:44:49
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-13 00:44:49
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:2(chi): Audio: ac3, 48000 Hz, stereo, fltp, 256 kb/s
Metadata:
BPS : 256000
BPS-eng : 256000
DURATION : 02:19:19.488000000
DURATION-eng : 02:19:19.488000000
NUMBER_OF_FRAMES: 261234
NUMBER_OF_FRAMES-eng: 261234
NUMBER_OF_BYTES : 267503616
NUMBER_OF_BYTES-eng: 267503616
_STATISTICS_WRITING_APP: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_DATE_UTC: 2017-02-13 00:44:49
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-13 00:44:49
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:3(eng): Subtitle: hdmv_pgs_subtitle (default)
Metadata:
BPS : 14213
BPS-eng : 14213
DURATION : 02:09:49.127000000
DURATION-eng : 02:09:49.127000000
NUMBER_OF_FRAMES: 3316
NUMBER_OF_FRAMES-eng: 3316
NUMBER_OF_BYTES : 13838703
NUMBER_OF_BYTES-eng: 13838703
_STATISTICS_WRITING_APP: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_DATE_UTC: 2017-02-13 00:44:49
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-13 00:44:49
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:4(chi): Subtitle: hdmv_pgs_subtitle
Metadata:
BPS : 23217
BPS-eng : 23217
DURATION : 02:13:03.100000000
DURATION-eng : 02:13:03.100000000
NUMBER_OF_FRAMES: 3423
NUMBER_OF_FRAMES-eng: 3423
NUMBER_OF_BYTES : 23168356
NUMBER_OF_BYTES-eng: 23168356
_STATISTICS_WRITING_APP: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_DATE_UTC: 2017-02-13 00:44:49
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-13 00:44:49
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:5(chi): Subtitle: hdmv_pgs_subtitle
Metadata:
BPS : 23882
BPS-eng : 23882
DURATION : 02:13:03.100000000
DURATION-eng : 02:13:03.100000000
NUMBER_OF_FRAMES: 3423
NUMBER_OF_FRAMES-eng: 3423
NUMBER_OF_BYTES : 23831726
NUMBER_OF_BYTES-eng: 23831726
_STATISTICS_WRITING_APP: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_DATE_UTC: 2017-02-13 00:44:49
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-13 00:44:49
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:6(chi): Subtitle: hdmv_pgs_subtitle
Metadata:
BPS : 22075
BPS-eng : 22075
DURATION : 02:13:03.100000000
DURATION-eng : 02:13:03.100000000
NUMBER_OF_FRAMES: 3423
NUMBER_OF_FRAMES-eng: 3423
NUMBER_OF_BYTES : 22028598
NUMBER_OF_BYTES-eng: 22028598
_STATISTICS_WRITING_APP: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_DATE_UTC: 2017-02-13 00:44:49
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-13 00:44:49
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:7(chi): Subtitle: hdmv_pgs_subtitle
Metadata:
BPS : 22543
BPS-eng : 22543
DURATION : 02:13:03.100000000
DURATION-eng : 02:13:03.100000000
NUMBER_OF_FRAMES: 3423
NUMBER_OF_FRAMES-eng: 3423
NUMBER_OF_BYTES : 22495941
NUMBER_OF_BYTES-eng: 22495941
_STATISTICS_WRITING_APP: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_DATE_UTC: 2017-02-13 00:44:49
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-13 00:44:49
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
Stream #0:0 -> #0:0 (hevc (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[vost#0:0/rawvideo @ 0x1482058c0] Finishing stream without any data written to it.
Output #0, rawvideo, to 'pipe:':
Metadata:
encoder : Lavf60.3.100
Chapters:
Chapter #0:0: start 0.000000, end 120.000000
Metadata:
title : Chapter 04
Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 1920x804 [SAR 1:1 DAR 160:67], q=2-31, 18524 kb/s, 0.50 fps, 0.50 tbn (default)
Metadata:
BPS : 5486317
BPS-eng : 5486317
DURATION : 02:19:19.435000000
DURATION-eng : 02:19:19.435000000
NUMBER_OF_FRAMES: 200426
NUMBER_OF_FRAMES-eng: 200426
NUMBER_OF_BYTES : 5732814506
NUMBER_OF_BYTES-eng: 5732814506
_STATISTICS_WRITING_APP: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v9.7.1 ('Pandemonium') 64bit
_STATISTICS_WRITING_DATE_UTC: 2017-02-13 00:44:49
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-13 00:44:49
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
encoder : Lavc60.3.100 rawvideo
frame= 0 fps=0.0 q=0.0 Lsize= 0kB time=-577014:32:22.77 bitrate= -0.0kbits/s speed=N/A
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
Exiting normally, received signal 15.
�[32m13:12:49�[0m :: �[1;35m FFdecoder �[0m :: �[1;33m DEBUG �[0m :: �[1;37mTerminating FFdecoder Pipeline...�[0m
�[32m13:12:49�[0m :: �[1;35m FFdecoder �[0m :: �[1;36m INFO �[0m :: �[1;37mPipeline already terminated.�[0m
Processed 0 frames, skipped_by_similarity: 0 frames
Python Code(Optional)
No response
DeFFcode Version
0.2.5
Python version
3.11
Operating System version
macOS M1 ( Darwin Kernel Version 23.2.0: Wed Nov 15 21:53:18 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T6000 arm64)
Any other Relevant Information?
No response
@humpydonkey Thanks for the issue, I didn't noticed it until now. Thankfully it's a easy fix. :)