Can't get latest ffmpeg logs due to non existent CreationTime / Birth Time for files in Linux
Closed this issue · 10 comments
jI am using an Artix based linux distro and I noticed the vmaf scores are all 0 in the log file when running the benchmark tool. The same occurs when running the permutor-cli tool without the -c flag.
Setup
- clone repo
- cargo run --bin benchmark -r
- cargo run --bin permutor-cli -r
- copy binaries from target/release to PATH
- copy source files into ssd
/usr/share/benchmark/encoding_files - cd into dir above and ran
benchmark/permutor-cli
Installed Dependencies
ffmpeg/ffprobe 6.0 (https://archlinux.org/packages/extra/x86_64/ffmpeg/)
Also tried these AUR packages, but got same error
Permutor CLI Stacktrace Info
When providing the permutor-cli tool with the -c flag, it errors out.
command: RUST_BACKTRACE=full permutor-cli -v -e h264_nvenc -s 4k-60.y4m -c -b 10 -m 100 -t
output
...
Average FPS: 49
1%'ile: 42
90%'ile: 46
Calculating vmaf score; might take longer than original encode depending on your CPU...
V: Vmaf args calculating quality: -y -report -r 60 -i tcp://localhost:2000?listen -r 60 -i 4k-60.y4m -filter_complex libvmaf='n_threads=12:n_subsample=5' -f null -
V: ffmpeg args: [-y -report -r 60 -i tcp://localhost:2000?listen -r 60 -i 4k-60.y4m -filter_complex libvmaf='n_threads=12:n_subsample=5' -f null -]
V: ffmpeg args no network calls (copy this and run locally, minus the quotes): [-y -report -r 60 -i tcp://localhost:2000?listen -r 60 -i 4k-60.y4m -filter_complex libvmaf='n_threads=12:n_subsample=5' -f null -]
V: Encoder fmmpeg args sending to vmaf: -progress tcp://localhost:1234 -stats_period 0.5 -y -i 4k-60.y4m -b:v 10M -c:v h264_nvenc -preset p1 -tune hq -profile:v high -rc cbr -cbr true -gpu 0 -f h264 tcp://localhost:2000
V: ffmpeg args: [-progress tcp://localhost:1234 -stats_period 0.5 -y -i 4k-60.y4m -b:v 10M -c:v h264_nvenc -preset p1 -tune hq -profile:v high -rc cbr -cbr true -gpu 0 -f h264 tcp://localhost:2000]
V: ffmpeg args no network calls (copy this and run locally, minus the quotes): [-y -i 4k-60.y4m -b:v 10M -c:v h264_nvenc -preset p1 -tune hq -profile:v high -rc cbr -cbr true -gpu 0 -f null -]
VMAF calculation finishing up...
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', ffmpeg/src/report_files.rs:24:91
...
Hmm the docs say:
pub fn from_creation_time(meta: &Metadata) -> Option<FileTime>
Creates a new timestamp from the creation time listed in the specified metadata.
The returned value corresponds to the birthtime field of stat on Unix platforms and the ftCreationTime field on Windows platforms. Note that not all Unix platforms have this field available and may return None in some circumstances.
I guess my Unix platform doesn't have this field, so it returns None
Hey there @e-dong! Thanks for opening this up, that's a weird issue I do see that in the docs as well 👀
A quick test would be maybe swapping that out for from_unix_time and re-running on your linux distro. If that doesn't fix it quick though, we can keep this issue open and I can investigate a bit more to get it fixed for you.
Interesting, I learned that some linux systems don't store the creation time. I wonder if I'm the first person trying this on linux lol.
I tried out this simple example from here and I can reproduce the error.
- alexcrichton/filetime#74
- https://stackoverflow.com/questions/76946130/creation-time-is-not-available-on-this-platform-currently
I'm new to rust, so I ended up using last_modification_time so I could maintain the same "metadata" parameter
while index != log_entries.len() {
entry = log_entries.get(index);
let file_time = FileTime::from_last_modification_time(&entry.unwrap().metadata().unwrap());
if file_time > latest_time {
latest_time = FileTime::from_last_modification_time(&entry.unwrap().metadata().unwrap());
log_file = entry;
}
...
Look like that is a workaround for now.
[eric@m3tis /usr/share/benchmark/encoding_files]$ permutor-cli -e h264_nvenc -s 4k-60.y4m -c -t
Options:
-calculating vmaf score
-test run, will only run 1 permutation
[ETR: Unknown until first permutation is done]
[Permutation: 1/1]
[Resolution: 3840x2160]
[Encoder: h264_nvenc]
[FPS: 60]
[Bitrate: 10Mb/s]
[-preset p1 -tune hq -profile:v high -rc cbr -cbr true -gpu 0]
[00:00:38] [####################################################################################################################################################] 1800/1800 frames (00:00:00)
Average FPS: 49
1%'ile: 44
90%'ile: 46
Calculating vmaf score; might take longer than original encode depending on your CPU...
[00:01:28] [####################################################################################################################################################] 1800/1800 frames (00:00:00)
VMAF calculation finishing up...
VMAF score: 62.93657
Benchmark runtime: 2m28s
@e-dong hehe you might be the first end-user using it on Linux I think yeah! xD that score looks correct for 10Mb/s so that's a good sign.
I can test using from_last_modification_time on both Windows and Linux, and push up that as the fix if it works on both! Appreciate you spending the time to test that out.
No problem, I'm a big fan of the work you done putting all of this together! I will add a guide on the Sunshine docs using your awesome tools!
@Proryanator aw spoke too soon I'm running into more errors. I will see how far I can get and report back to you.
Ohh so I switched back to the official arch version of ffmpeg and it works now! I was getting lower than normal fps values @4k (I have a RTX 3070) before with the AUR ffmpeg-full package.
The tcp connection was timing out or something so the command just hanged after trying to finish the cal of vmaf score. When I Ctrl +c I realize the error I got was because the script was reading
tcp @ 0x563aed8a2480] No default whitelist set which doesn't match the regex to get the vmaf score lol.
I made a PR with my fork, I also plan to add some Linux setup instructions in the readme.
Sweet thanks! Looks good, I'll probably build the tool and try it out on Windows, then I can get your PR in.
I was getting lower than normal fps values @4k (I have a RTX 3070)
Oops my sequential read speed is at like 525 MB/s lol.
Learned you can do this easily in Linux
https://www.baeldung.com/linux/disk-performance-test