how to encode and output file from anime4k？

Question

how to encode and output file from anime4k？

lsl330 opened this issue 3 years ago · 87 comments

I have no problem playing anime4k video via mpv, and I have no problem playing it via command line, but I want to output the video to 1080p, how should I achieve that?

command
mpv input.mkv --no-sub --vo=gpu --glsl-shaders="~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Restore_CNN_VL.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_VL.glsl;~~/shaders/Anime4K_Restore_CNN_M.glsl;~~/shaders/Anime4K_AutoDownscalePre_x2.glsl;~~/shaders/Anime4K_AutoDownscalePre_x4.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl"
it work and play as well

but command
mpv input.mkv --no-sub --vo=gpu --glsl-shaders="~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Restore_CNN_VL.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_VL.glsl;~~/shaders/Anime4K_Restore_CNN_M.glsl;~~/shaders/Anime4K_AutoDownscalePre_x2.glsl;~~/shaders/Anime4K_AutoDownscalePre_x4.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl" -vf scale=1440:1080 --o=output.mkv
No effect，
how can i set the Resolution to 1080p and output the anime4k video？

Answer 1 · 2021-12-12T01:41:39.000Z

~~no meaning of using vo=gpu whiling encoding....~~
To be clear, do not use it.
vf=gpu="w=1920:h=1080"

https://mpv.io/manual/master/#video-filters-gpu

Answer 2 · 2021-12-12T13:35:28.000Z

no meaning of using vo=gpu whiling encoding.... To be clear, do not use it. vf=gpu="w=1920:h=1080"

https://mpv.io/manual/master/#video-filters-gpu

thank you for reply,but when i use
vf=gpu="w=1920:h=1080"
it have an error,
Option vf: gpu doesn't exist.
Error parsing option vf (option parameter could not be parsed)
Setting commandline option --vf=gpu=w=1440:h=1080 failed.

Answer 3 · 2021-12-12T13:41:25.000Z

This requires that the EGL implementation supports off-screen rendering on the default display. (This is the case with Mesa.)

See manual. windows does not support now.
#9354

Answer 4 · 2021-12-12T13:53:23.000Z

This requires that the EGL implementation supports off-screen rendering on the default display. (This is the case with Mesa.)

See manual. windows does not support now. #9354

thank you, it seems that I can only wait until later to add support for Win

Answer 5 · 2021-12-12T15:55:57.000Z

If your ffmpeg is built with the corresponding options, you can use the vf_libplacebo filter in ffmpeg to apply shaders, including Anime4K, while transcoding.

Answer 6 · 2021-12-13T00:29:23.000Z

If your ffmpeg is built with the corresponding options, you can use the vf_libplacebo filter in ffmpeg to apply shaders, including Anime4K, while transcoding.

more detial？
标语言英语（美式）
Can you please give me the full command line?

Answer 7 · 2021-12-25T11:10:43.000Z

I compiled the latest ffmpeg commit with libplacebo filter, but i'm getting this error when trying to add it to my command
ffmpeg -y -i clip.avs -vf libplacebo=custom_shader_path=Anime4K_Thin_HQ.glsl -vcodec libx264 -an out.mp4

[avisynth @ 000001c5f71da4c0] Stream #0: not enough frames to estimate rate; consider increasing probesize
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, avisynth, from 'clip.avs':
Duration: 00:00:05.84, start: 0.000000, bitrate: 1 kb/s
Stream #0:0: Video: rawvideo (Y3[11][10] / 0xA0B3359), yuv420p10le, 1920x1080, 23.98 fps, 23.98 tbr, 23.98 tbn
Stream #0:1: Audio: pcm_s32le, 48000 Hz, stereo, s32, 3072 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:0' and the filter 'auto_scale_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
Conversion failed!

Answer 8 · 2021-12-25T20:08:40.000Z

@lsl330 @aypro it's implemented as a vulkan filter so you have to upload the frame to vulkan first:

ffmpeg -i CLIP -init_hw_device vulkan -vf hwupload,libplacebo,hwdownload -vcodec libx264 OUT

Answer 9 · 2021-12-26T10:52:52.000Z

Thank you @haasn , this is what ended up working for me, just had to explicitly add the format on both ends, also i don't know if this is the correct way to apply multiple shaders
ffmpeg -y -i IN.MKV -init_hw_device vulkan -vf format=yuv420p,hwupload,libplacebo=custom_shader_path=Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=Anime4K_Restore_CNN_Moderate_VL.glsl,libplacebo=custom_shader_path=Anime4K_Darken_HQ.glsl,libplacebo=custom_shader_path=Anime4K_Thin_HQ.glsl,hwdownload,format=yuv420p -vcodec libx264 -an out.mp4

Answer 10 · 2021-12-26T14:03:03.000Z

That's a bit wasteful, it would be much better to cat all shaders into the same file and load that. (Though I'll probably fix vf_libplacebo to accept multiple shaders soon-ish)

That said, to make it significantly less wasteful, you could add format=rgba (or rgba64le) to the first libplacebo filter and format=yuv420p to the last libplacebo filter. (That way you at least avoid constantly converting between rgb and yuv420p)

Answer 11 · 2022-03-22T05:31:43.000Z

I have problem when I use bat to upscale video.

However something was wrong in cmd:
Trailing option(s) found in the command: may be ignored.
Device creation failed: -12.
Failed to set value 'vulkan' for option 'init_hw_device': Cannot allocate memory
Error parsing global options: Cannot allocate memory

Answer 12 · 2022-03-22T10:36:15.000Z

Thank you @haasn , this is what ended up working for me, just had to explicitly add the format on both ends, also i don't know if this is the correct way to apply multiple shaders ffmpeg -y -i IN.MKV -init_hw_device vulkan -vf format=yuv420p,hwupload,libplacebo=custom_shader_path=Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=Anime4K_Restore_CNN_Moderate_VL.glsl,libplacebo=custom_shader_path=Anime4K_Darken_HQ.glsl,libplacebo=custom_shader_path=Anime4K_Thin_HQ.glsl,hwdownload,format=yuv420p -vcodec libx264 -an out.mp4

Can you upscale video by this command successfully? There's something wrong when I use the command. the error in cmd:
"Trailing option(s) found in the command: may be ignored.
Device creation failed: -12.
Failed to set value 'vulkan' for option 'init_hw_device': Cannot allocate memory"
I'm not good at ffmpeg, and I'm not sure if it's the problem of command "-init_hw_device vulkan".

Answer 13 · 2022-03-23T01:57:13.000Z

check if your local build support vulkan ffmpeg -init_hw_device list

Answer 14 · 2022-03-23T02:16:53.000Z

check if your local build support vulkan ffmpeg -init_hw_device list

Supported hardware device types:
cuda
dxva2
qsv
d3d11va

Answer 15 · 2022-03-23T02:18:35.000Z

So, obviously no.

Answer 16 · 2022-03-23T02:20:41.000Z

Can I use "cuda" instead of "vulkan"?
Or how can I output enlarged video by other way?

So, obviously no.

Answer 17 · 2022-03-23T04:31:36.000Z

That's a bit wasteful, it would be much better to cat all shaders into the same file and load that. (Though I'll probably fix vf_libplacebo to accept multiple shaders soon-ish)

That said, to make it significantly less wasteful, you could add format=rgba (or rgba64le) to the first libplacebo filter and format=yuv420p to the last libplacebo filter. (That way you at least avoid constantly converting between rgb and yuv420p)

My ffmpeg(v5.0) build doesn't support vulkan, how to solve the problem or if there's anyway else to achieve outputting videos from Anime4K? And I also have a question about ffmpeg -i CLIP -init_hw_device vulkan -vf hwupload,libplacebo,hwdownload -vcodec libx264 OUT, "libplacebo" is used to set custom path of filter, right?

Answer 18 · 2022-05-15T04:13:49.000Z

With a standard Linux installation of GLSL for MPV, here's the full command I used to render the output. It works like a charm. IIRC this should be using the same shaders as Mode A.

mpv input.mp4 --no-sub --glsl-shaders="~/.config/mpv/shaders/Anime4K_Clamp_Highlights.glsl:~/.config/mpv/shaders/Anime4K_Restore_CNN_VL.glsl:~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_VL.glsl:~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x2.glsl:~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x4.glsl:~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_M.glsl" -vf=gpu="w=3840:h=2160" --o=output.mkv

Edit: this should be easier to read.

mpv input.mp4 \
    --no-sub \
    --glsl-shaders="\
~/.config/mpv/shaders/Anime4K_Clamp_Highlights.glsl:\
~/.config/mpv/shaders/Anime4K_Restore_CNN_VL.glsl:\
~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_VL.glsl:\
~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x2.glsl:\
~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x4.glsl:\
~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_M.glsl" \
    -vf=gpu="w=1216:h=2160" \
    --o=output.mkv

Answer 19 · 2022-07-15T01:03:06.000Z

@k4yt3x There's a mistake in your last command, the width should be 3840, not 1216

Answer 20 · 2022-07-18T00:17:09.000Z

@k4yt3x There's a mistake in your last command, the width should be 3840, not 1216

That's for the specific video I'm processing. Just do the calculations and change it to whatever you need it to be.

Answer 21 · 2022-07-25T06:30:24.000Z

This requires that the EGL implementation supports off-screen rendering on the default display. (This is the case with Mesa.)

See manual. windows does not support now. #9354

现在在windows上不能使用--vf=gpu的话是不是就无法保存anime4k放大的视频了？Windows在运行下面的命令时候总是会报错：mpv input.mp4 --glsl-shaders="%appdata%/mpv/shaders/Anime4K_Clamp_Highlights.glsl" --vf=gpu -vf scale="1920:1080" --o=output.mkv
报错：Option vf: gpu doesn't exist.

Answer 22 · 2022-07-25T08:56:45.000Z

it's --vo=gpu

Answer 23 · 2022-07-25T10:32:56.000Z

报错：Option vf: gpu doesn't exist.

Because there is no implement in windows. #9589 (comment)

it's --vo=gpu

No, it's vf not vo. It's clear in the doc and previous comments.

Answer 24 · 2022-09-27T23:29:10.000Z

I encoded video using this command:
mpv input.mp4 --glsl-shader="~~/shaders/Anime4K_Clamp_Highlights.glsl:~~/shaders/Anime4K_Restore_CNN_VL.glsl:~~/shaders/Anime4K_Upscale_CNN_x2_VL.glsl:~~/shaders/Anime4K_AutoDownscalePre_x2.glsl:~~/shaders/Anime4K_AutoDownscalePre_x4.glsl:~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl" -vf scale="2560:1440" -o output.mp4

I got output file 1440p resolution, but it looks shaders are not applied - output file looks same as input. It just scaled file.
When I use vf=gpu nothing happens.

Did anyone find way to encode video with Anime4K shaders?

Platform: Windows

Answer 25 · 2022-09-28T00:16:31.000Z

Did anyone find way to encode video with Anime4K shaders?

FFmpeg with vf_libplacebo is the recommended way.

Answer 26 · 2022-09-28T05:11:19.000Z

Did anyone find way to encode video with Anime4K shaders?

FFmpeg with vf_libplacebo is the recommended way.

Already tried all ways from this thread, still getting errors.
Can you share correct parameters for ffmpeg?

Answer 27 · 2022-10-29T05:03:03.000Z

On Windows:

Download ffmpeg build from https://github.com/BtbN/FFmpeg-Builds/releases

./ffmpeg.exe -y -i input.mp4 -init_hw_device vulkan -vf scale=3840:2160,format=yuv420p,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_M.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x2.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x4.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_M.glsl,hwdownload,format=yuv420p -vcodec libx264 out.mp4

Modify the scale and formats accordingly.

There's something wonky with libplacebo at the moment not liking a particular statement in the glsl filters.

Remove //!SAVE MAIN from all the glsl filters otherwise nothing will happen.

Tested this on Windows 11 with a 3080 Ti.

EDIT:

After a bit more testing, I can now tell that certain filters are being applied, and certain filters are not. There's still a discernable difference in certain files, not exactly sure where the issue is.

Answer 28 · 2022-10-30T01:27:54.000Z

On Windows:

Download ffmpeg build from https://github.com/BtbN/FFmpeg-Builds/releases

./ffmpeg.exe -y -i input.mp4 -init_hw_device vulkan -vf scale=3840:2160,format=yuv420p,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_M.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x2.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x4.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_M.glsl,hwdownload,format=yuv420p -vcodec libx264 out.mp4

Modify the scale and formats accordingly.

There's something wonky with libplacebo at the moment not liking a particular statement in the glsl filters.

Remove //!SAVE MAIN from all the glsl filters otherwise nothing will happen.

Tested this on Windows 11 with a 3080 Ti.

EDIT:

After a bit more testing, I can now tell that certain filters are being applied, and certain filters are not. There's still a discernable difference in certain files, not exactly sure where the issue is.

This worked pretty well for me with some small tweaks and simplified shaders.

I merged all the shaders for each mode into files using
copy filter1.glsl + filter2.glsl + filteretc... "outputFilterName.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeA.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_Soft_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4k_ModeB.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Upscale_Denoise_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_modeC.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_Restore_CNN_M.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeA+A.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_Soft_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Restore_CNN_Soft_M.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeB+B.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Upscale_Denoise_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Restore_CNN_M.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeC+A.glsl"

I then removed all the //!SAVE MAIN from the merged glsl files

For anyone interested in downloading these merged shaders here they are:
shaders.zip

I then used PowerShell to run FFmpeg with the following command

./ffmpeg.exe -y -i "inputVideo.mp4" -init_hw_device vulkan -vf scale=3840:2160:flags=lanczos,format=yuv420p10,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -vcodec libx264 "outputVideo.mkv"

The flags=lanczos is what made the biggest difference for me, without it the final output had thicker lines and was not nearly as sharp as it was compared to mpv. https://write.corbpie.com/a-guide-to-upscaling-or-downscaling-video-with-ffmpeg/#:~:text=Scaling%20with%20a%20video%20scaling%20algorithm did a good job explaining the possible flags options.

Original Image:

anime4k mode A filter in MPV:

anime4k mode A filter ffmpeg:

anime4k mode A filter ffmpeg without flags=lanczos:

I do apologize if the images are not full quality because they were compressed to upload to github.
I also apologize if the comparison is a little annoying because they are not on the same frame. I was a little lazy.

There may be some differences between the pictures but they are small enough that I did not notice them while watching a short clip. I also use "ewa_lanczossharp" in mpv which is slightly different from lanczos

This is the mpv.conf file I used for the anime4k mode A filter in MPV

profile=gpu-hq
save-position-on-quit=yes
scale=ewa_lanczossharp
cscale=ewa_lanczossharp
alang=jpn,eng
slang=eng,jpn
--sub-blur=0.3
glsl-shaders="~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Restore_CNN_VL.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_VL.glsl;~~/shaders/Anime4K_AutoDownscalePre_x2.glsl;~~/shaders/Anime4K_AutoDownscalePre_x4.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl"

Answer 29 · 2022-10-30T14:22:45.000Z

@ryandash I tried that but I just get

Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
Assertion failed: sem_out.sem, file ../src/vulkan/gpu_tex.c, line 1121

Answer 30 · 2022-10-30T15:52:49.000Z

@ryandash I tried that but I just get

Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
Assertion failed: sem_out.sem, file ../src/vulkan/gpu_tex.c, line 1121

Try this

./ffmpeg.exe -y -i "inputVideo.mp4" -init_hw_device vulkan -vf scale=3840:2160:flags=lanczos,format=yuv420p,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p -vcodec libx264 "outputVideo.mkv"

I assume you are using an AMD GPU. I only have an nvidia gpu to test with and there does seem to be known issues with AMD GPUs when converting with ffmpeg on Windows. If there are still issues then you can try https://ubuntu.com/wsl which let's you use Linux on windows. The Linux mesa driver for AMD GPUs seems to be more stable.

Answer 31 · 2022-10-30T15:56:23.000Z

If there are still issues then you can try https://ubuntu.com/wsl

Use a standard Linux installation is better, wsl may meet the problem of lacking of EGL.

Answer 32 · 2022-10-30T16:13:45.000Z

Does anybody know recipe to make it working at macos? Or at least at ubuntu, because after some hours I struggle now at

ffmpeg -i input.ts -init_hw_device vulkan  outputVideo.mkv
[AVHWDeviceContext @ 0x562ed6034a00] Device does not support timeline semaphores!
Device creation failed: -38.
Failed to set value 'vulkan' for option 'init_hw_device': Function not implemented

while trying to make it work at cloud VM with ubuntu 20.

Answer 33 · 2022-10-30T16:22:58.000Z

ref the previous comment, check your local ffmpeg first #9589 (comment)

Answer 34 · 2022-10-30T16:25:21.000Z

It supports as I see

ffmpeg -init_hw_device list
ffmpeg version 5.1.2-0ubuntu1~20.04.sav1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
  configuration: --prefix=/usr --extra-version='0ubuntu1~20.04.sav1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-lcms2 --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-sndio --enable-pocketsphinx --enable-librsvg --enable-libdav1d --enable-libjxl --enable-librist --enable-libvmaf --enable-libzimg --enable-crystalhd --enable-libmfx --enable-libsvtav1 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Supported hardware device types:
vdpau
cuda
vaapi
qsv
drm
opencl
vulkan

Answer 35 · 2022-10-30T16:27:45.000Z

make it working at macos

What you post is linux, not mac.

Answer 36 · 2022-10-30T16:32:12.000Z

Yeah, i know, it is ubuntu at cloud, my mac supports only videotoolbox, so i stopped my attempts with it, and trying with that ubuntu

Answer 37 · 2022-10-30T18:15:35.000Z

On Windows:
Download ffmpeg build from https://github.com/BtbN/FFmpeg-Builds/releases
./ffmpeg.exe -y -i input.mp4 -init_hw_device vulkan -vf scale=3840:2160,format=yuv420p,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_M.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x2.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x4.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_M.glsl,hwdownload,format=yuv420p -vcodec libx264 out.mp4
Modify the scale and formats accordingly.
There's something wonky with libplacebo at the moment not liking a particular statement in the glsl filters.
Remove //!SAVE MAIN from all the glsl filters otherwise nothing will happen.
Tested this on Windows 11 with a 3080 Ti.
EDIT:
After a bit more testing, I can now tell that certain filters are being applied, and certain filters are not. There's still a discernable difference in certain files, not exactly sure where the issue is.

This worked pretty well for me with some small tweaks and simplified shaders.

I merged all the shaders for each mode into files using copy filter1.glsl + filter2.glsl + filteretc... "outputFilterName.glsl"
copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeA.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_Soft_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4k_ModeB.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Upscale_Denoise_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_modeC.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_Restore_CNN_M.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeA+A.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_Soft_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Restore_CNN_Soft_M.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeB+B.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Upscale_Denoise_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Restore_CNN_M.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeC+A.glsl"
I then removed all the //!SAVE MAIN from the merged glsl files

For anyone interested in downloading these merged shaders here they are: shaders.zip

I then used PowerShell to run FFmpeg with the following command

./ffmpeg.exe -y -i "inputVideo.mp4" -init_hw_device vulkan -vf scale=3840:2160:flags=lanczos,format=yuv420p10,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -vcodec libx264 "outputVideo.mkv"

The flags=lanczos is what made the biggest difference for me, without it the final output had thicker lines and was not nearly as sharp as it was compared to mpv. https://write.corbpie.com/a-guide-to-upscaling-or-downscaling-video-with-ffmpeg/#:~:text=Scaling%20with%20a%20video%20scaling%20algorithm did a good job explaining the possible flags options.

Original Image:

anime4k mode A filter in MPV:

anime4k mode A filter ffmpeg:

anime4k mode A filter ffmpeg without flags=lanczos:

I do apologize if the images are not full quality because they were compressed to upload to github. I also apologize if the comparison is a little annoying because they are not on the same frame. I was a little lazy.

There may be some differences between the pictures but they are small enough that I did not notice them while watching a short clip. I also use "ewa_lanczossharp" in mpv which is slightly different from lanczos

This is the mpv.conf file I used for the anime4k mode A filter in MPV
profile=gpu-hq
save-position-on-quit=yes
scale=ewa_lanczossharp
cscale=ewa_lanczossharp
alang=jpn,eng
slang=eng,jpn
--sub-blur=0.3
glsl-shaders="~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Restore_CNN_VL.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_VL.glsl;~~/shaders/Anime4K_AutoDownscalePre_x2.glsl;~~/shaders/Anime4K_AutoDownscalePre_x4.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl"

Did all the modes work correctly for you? I noticed that certain filters weren't being applied. I compared the footage that was encoded with the original video playing in MPV with the same modes and could immediately tell differences.

Answer 38 · 2022-10-30T20:18:40.000Z

There are definitely some differences when compared side by side for some modes. From my testing, Modes A, A + A, C, and C + A had minor differences between screenshots taken from FFmpeg and MPV but there are differences. Modes B and B+B have much more obvious differences that are noticeable. I am not sure what is causing the differences but it may be that while removing //!SAVE MAIN allows for the shaders to be applied, they are not being applied properly because //!SAVE MAIN was removed or it may be something else entirely. The issue is NOT resolved because the output of FFmpeg does not create the exact same output as MPV. When I said "this worked pretty well for me", I meant that I was content with the final output even if it is not the exact same.

Answer 39 · 2022-10-31T01:18:50.000Z

[AVHWFramesContext @ 0000024d36f98180] Cannot allocate all planes in a single allocation, device requires dedicated image allocation! [hwupload @ 0000024d36fb9180] Failed to allocate frame to upload to. Error while filtering: Cannot allocate memory Failed to inject frame into filter network: Cannot allocate memory Error while processing the decoded data for stream #0:0 [aac @ 0000024d2ef0d5c0] Qavg: 1218.201 [aac @ 0000024d2ef0d5c0] 2 frames left in the queue on closing Conversion failed!
The error from command ./ffmpeg.exe -y -i input.mp4 -init_hw_device vulkan -vf scale=3840:2160,format=yuv420p,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_M.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x2.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x4.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_M.glsl,hwdownload,format=yuv420p -vcodec libx264 out.mp4
Is there anything which I should change in this command?

Answer 40 · 2022-10-31T01:39:53.000Z

I plan on expanding vf_libplacebo to support software formats as well, which should solve most of these issues

Answer 41 · 2022-10-31T13:15:32.000Z

@ryandash I tried that but I just get
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
Assertion failed: sem_out.sem, file ../src/vulkan/gpu_tex.c, line 1121
Try this

./ffmpeg.exe -y -i "inputVideo.mp4" -init_hw_device vulkan -vf scale=3840:2160:flags=lanczos,format=yuv420p,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p -vcodec libx264 "outputVideo.mkv"

I assume you are using an AMD GPU. I only have an nvidia gpu to test with and there does seem to be known issues with AMD GPUs when converting with ffmpeg on Windows. If there are still issues then you can try https://ubuntu.com/wsl which let's you use Linux on windows. The Linux mesa driver for AMD GPUs seems to be more stable.

Yeah I'm on an RX570 and even with yuv420p the same thing happens. Don't really need it enough to bother with WSL right now, rather just wait for it to get fixed.

Answer 42 · 2022-11-04T16:54:59.000Z

Btw, I fixed the //!SAVE MAIN bug on master. But it unfortunately doesn't explain the differences, because simply removing it had the same effect in these cases.

Answer 43 · 2022-11-04T16:57:47.000Z

Is it possible that we are seeing differences due to the FBO format? --vo=gpu-next currently always uses rgba16hf for FBOs, and this is not really changeable. But I think --vo=gpu defaults to rgba16.

Can you try playing around with --vo=gpu --fbo-format=rgba16hf and --vo=gpu --fbo-format=rgba32f to see if it changes the results?

Answer 44 · 2022-11-04T17:01:15.000Z

But I think --vo=gpu defaults to rgba16.

It's up to gpu-api . rgba16hf is the default value for d3d11

if it changes the results

I didn't test more samples. Checked the output here I couldn't see any diff (The screenshots even share the same hash).

Answer 45 · 2022-11-15T01:36:27.000Z

I think I found an easy solution but I am not sure why it has not already been tried before now.
pr #10102 allowed me to use vf=gpu in windows.
While pr #10102 may have other issues that prevent it from being merged, I did not have any issues with my limited testing of encoding a video other than a basic merging issue between the pr and master branch.

I merged pr #10102 (#10102) into the master branch on my local git, fixed the merging issues (basic issues with log in mpv\video\filter\vf_gpu_vulkan.c and meson.build), and compiled MPV using MSYS2 following the instructions here https://github.com/mpv-player/mpv/blob/master/DOCS/compile-windows.md with minor tweaks to enable Vulkan.

I then used the newly made mpv and the following command:
mpv "input.mkv" -glsl-shaders="Anime4K_ModeA.glsl" -vf=gpu="w=3840:h=2160" -scale=ewa_lanczossharp -cscale=ewa_lanczossharp --o="test.mkv"

On the latest master of mpv using -vf=gpu on windows would not work but it worked just fine with this build.

Here is a small clip before and after:
original:
https://user-images.githubusercontent.com/56605453/201801723-4a477bce-b642-429e-a078-43616695a401.mp4

Anime4k encode
https://user-images.githubusercontent.com/56605453/201802067-80b36298-f6d3-494c-bb0a-c49303337014.mp4

Here is the compiled mpv if you are willing to download it and try it out. mpv download through google drive Be aware that I included a bunch of unnecessary files in the same folder because this compiled version of mpv requires a lot of them to run unlike the official one and I am too lazy to figure out which ones it needs. I highly recommend that you compile it yourself following the basic changes I made above as I do not trust files (especially executable programs) from random people.

Here are some screenshots showing the differences (I used google drive to avoid compressing them)
screenshots on google drive

Edit:
Updated download link with all shaders

Answer 46 · 2022-11-15T21:10:00.000Z

@ryandash Do you mind creating a fork of that linked PR with the fixes you made and how you managed to compile all the dependencies like libarchive into DLLs in the build dir?

Edit: I got it working (it would still be useful to share how you did it), but I still had to copy all the DLLs from your bin directory in. Thanks a lot for sharing this so far, this is great for me as I run a powerful Windows PC but a relatively weak Linux media server.

Answer 47 · 2022-11-16T00:15:30.000Z

@ryandash Do you mind creating a fork of that linked PR with the fixes you made and how you managed to compile all the dependencies like libarchive into DLLs in the build dir?

Edit: I got it working (it would still be useful to share how you did it), but I still had to copy all the DLLs from your bin directory in. Thanks a lot for sharing this so far, this is great for me as I run a powerful Windows PC but a relatively weak Linux media server.

Here are the steps I took to compile it

download and install https://www.msys2.org/
Close popup windows after installation.
go to C:\msys64 (default installation of msys64)
Start a MinGW-w64 shell (mingw64.exe)

pacman -Syu
pacman -Su
pacman -S git python $MINGW_PACKAGE_PREFIX-{pkg-config,gcc}
pacman -S $MINGW_PACKAGE_PREFIX-{ffmpeg,libjpeg-turbo,lua51,libplacebo}
git clone -b vf-gpu-vk https://github.com/tantei3/mpv.git
cd mpv
/usr/bin/python3 bootstrap.py
/usr/bin/python3 waf configure CC=gcc.exe --check-c-compiler=gcc --prefix=$MSYSTEM_PREFIX
/usr/bin/python3 waf install

find mpv file in 'msys64\mingw64\bin'

Answer 48 · 2022-11-19T15:10:39.000Z

@ryandash I haven't tested this but that looks about on-par with what we expect in real time encoding.

I think mostly people aren't looking at the PRs to find fixes, I know I didn't 😆

Awesome job.

That being said, I think we were looking at it more from a libplacebo angle since in theory it's supposed to output the same video as mpv but apparently that's still not the case.

Answer 49 · 2022-11-29T21:22:01.000Z

@ryandash Hi, I tried your build of mpv, but I got error while compiling shaders (bundled in bin folder).
Logs: https://pastebin.com/Ev954x2j
Do you have any idea to fix this?

//EDIT: I found solution, at the bottom of bundled shader there's some weird ASCII character (0x1A - SUB) of the file. Simply remove it in any text editor app

Answer 50 · 2023-01-19T23:34:50.000Z

@Dudemanguy I am not sure if this issue is really considered resolved until a change has been added to the mpv master branch or libplacebo creates a proper output using FFmpeg. What I have done is create a working sample that I will not update which may cause issues for people in the future. If pr #10102 is fixed up, merged with master and works then this issue will be resolved.

Answer 51 · 2023-01-20T00:02:52.000Z

Oh I actually missed that apparently the libplacebo output isn't exactly the same. Still, this wasn't really an issue or even feature request (we have one already for vf=gpu on windows) but just a question. I think any issues you might be having with vf_libplacebo would be best left in the proper channels.

Answer 52 · 2023-02-20T08:53:19.000Z

Does anybody know recipe to make it working at macos? Or at least at ubuntu, because after some hours I struggle now at
ffmpeg -i input.ts -init_hw_device vulkan  outputVideo.mkv
[AVHWDeviceContext @ 0x562ed6034a00] Device does not support timeline semaphores!
Device creation failed: -38.
Failed to set value 'vulkan' for option 'init_hw_device': Function not implemented
while trying to make it work at cloud VM with ubuntu 20.

Have you solved it? I meet the same problem in windows 10.

Answer 53 · 2023-02-24T06:25:41.000Z

Have you solved it? When I ran cmd-“ffmpeg -i hdr2sdr.mp4 -init_hw_device vulkan -vf hwupload,libplacebo=format=yuv420p,hwdownload,format=yuv420p hdr.mp4” I got the same error

Answer 54 · 2023-03-06T10:01:03.000Z

Does anybody know recipe to make it working at macos? Or at least at ubuntu, because after some hours I struggle now at
ffmpeg -i input.ts -init_hw_device vulkan  outputVideo.mkv
[AVHWDeviceContext @ 0x562ed6034a00] Device does not support timeline semaphores!
Device creation failed: -38.
Failed to set value 'vulkan' for option 'init_hw_device': Function not implemented
while trying to make it work at cloud VM with ubuntu 20.
Have you solved it? I meet the same problem in windows 10.

I had same error and solved it by installing newer version of ffmpeg from here.

Answer 55 · 2023-04-01T17:22:58.000Z

Here is a solution for anyone interested in encoding using anime4k shaders in FFmpeg.

For CPU encoding:

ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 "output.mkv"

For nvenc encoding:

ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -c:v hevc_nvenc -cq 24 -bf 5 -refs 5 -preset p7 "output.mkv"

Shaders that I used: (Edit updated shaders)
shaders.zip

This can probably be done with the original shaders but I am not going to bother testing something that I am not going to use.

Answer 56 · 2023-04-03T10:34:47.000Z

Here is a solution for anyone interested in encoding using anime4k shaders in FFmpeg.

For CPU encoding:

ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 "output.mkv"

For nvenc encoding:

ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -c:v hevc_nvenc -cq 24 -bf 5 -refs 5 -preset p7 "output.mkv"

Shaders that I used: shaders.zip

This can probably be done with the original shaders but I am not going to bother testing something that I am not going to use.

only modeA can be used，the error is in the last line

Answer 57 · 2023-04-03T12:41:59.000Z

Here is a solution for anyone interested in encoding using anime4k shaders in FFmpeg.
For CPU encoding:
ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 "output.mkv"
For nvenc encoding:
ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -c:v hevc_nvenc -cq 24 -bf 5 -refs 5 -preset p7 "output.mkv"
Shaders that I used: shaders.zip
This can probably be done with the original shaders but I am not going to bother testing something that I am not going to use.

only modeA can be used，the error is in the last line

Yep, thanks for the quick catch. I will update the shaders

Answer 58 · 2023-05-04T21:22:51.000Z

Yep, thanks for the quick catch. I will update the shaders
Hello i keep getting:
[libplacebo @ 0000015a5016e3c0] Missing device feature: dynamicRendering
[libplacebo @ 0000015a5016e3c0] Imported Vulkan device was not created with all required features!
[libplacebo @ 0000015a5016e3c0] Failed importing vulkan device
[libplacebo @ 0000015a5016e3c0] Failed importing vulkan device to libplacebo!
[Parsed_libplacebo_2 @ 0000015a4bad60c0] Query format failed for 'Parsed_libplacebo_2': Generic error in an external library
Error reinitializing filters!
Failed to inject frame into filter network: Generic error in an external library
Error while processing the decoded data for stream #0:0
Conversion failed!

This error.

I have gtx 1650 and it says it supports vulkan

Answer 59 · 2023-05-04T21:54:54.000Z

It would be nice if there was a common place/thread where we can report issues related to this, but unfortunately for now this GitHub issue seems like the only place even if it's not mpv related so I hope it doesn't get locked.

Hello i keep getting:

[libplacebo @ 0000015a5016e3c0] Missing device feature: dynamicRendering
[libplacebo @ 0000015a5016e3c0] Imported Vulkan device was not created with all required features!
[libplacebo @ 0000015a5016e3c0] Failed importing vulkan device
[libplacebo @ 0000015a5016e3c0] Failed importing vulkan device to libplacebo!

This error.

I have gtx 1650 and it says it supports vulkan

@Sanaan01 I just got the same error today on my RTX 4090 with the latest NVIDIA game-ready driver (531.79) after building the latest libplacebo myself. Looks like this was added 2 weeks ago. I fixed it by using an older libplacebo version, i.e. pacman -S mingw-w64-x86_64-libplacebo and building ffmpeg against this.

cc @haasn (might require an issue report there but it doesn't bother me since the old libplacebo version works fine)

Answer 60 · 2023-05-04T22:03:01.000Z

Yep, thanks for the quick catch. I will update the shaders
Hello i keep getting:
[libplacebo @ 0000015a5016e3c0] Missing device feature: dynamicRendering
[libplacebo @ 0000015a5016e3c0] Imported Vulkan device was not created with all required features!
[libplacebo @ 0000015a5016e3c0] Failed importing vulkan device
[libplacebo @ 0000015a5016e3c0] Failed importing vulkan device to libplacebo!
[Parsed_libplacebo_2 @ 0000015a4bad60c0] Query format failed for 'Parsed_libplacebo_2': Generic error in an external library
Error reinitializing filters!
Failed to inject frame into filter network: Generic error in an external library
Error while processing the decoded data for stream #0:0
Conversion failed!

This error.

I have gtx 1650 and it says it supports vulkan

Is that 1650 in a laptop or desktop?
What OS are you running?
Is your OS up to date?
Do you have integrated and dedicated graphics?
Are your drivers up to date?
Are you using the latest version of ffmpeg?
What command are you using with ffmpeg?
Are you even trying to use ffmpeg?
I don't mean to be the devil but all you have given us is errors and basically useless info... Can you provide more information that will actually help understand what the issue could be?

What you have said is.. I have errors and this is my model of graphics card...

If you have a laptop with integrated and dedicated graphics. This might be a solution to your problem. IDK

https://www.reddit.com/r/vulkan/comments/qsthqq/enable_vulkan_for_gtx_1650_laptop/

Answer 61 · 2023-05-04T22:17:33.000Z

I don't mean to be the devil but all you have given us is errors and basically useless info... Can you provide more information that will actually help understand what the issue could be?

Sorry for not providng more info

the script i use was:
ffmpeg -y -vsync 0 -i "ASilentVoice(KoenoKatachi).mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA+A.glsl,hwdownload,format=yuv420p10 "A Silent Voice (Koe no Katachi)4K2.mkv"

i am using the latest version of ffmpeg
I have a desktop 1650
Tried it after updating my drivers still same error
OS is up to date running windows 11
@FarisR99 I'm going to attempt to to do what you said to do may you help me I am new to ffmpeg hos do I build it using and older version of libplacebo

Answer 62 · 2023-05-04T22:30:37.000Z

@Sanaan01 If you're using a prebuilt ffmpeg binary, try looking for an older release from a month ago. Much easier than building it yourself - though I do have documentation on building ffmpeg and mpv yourself if needed.

Answer 63 · 2023-05-04T22:33:03.000Z

Okay will try that when I downloaded ffmpeg i just used the latest release @FarisR99 march 20th build is fine i presume

Answer 64 · 2023-05-04T22:43:02.000Z

Thank you both for your time it works now I assume I should use CPU encoder to give better quality right?

Answer 65 · 2023-05-05T00:08:17.000Z

I should use CPU encoder to give better quality right?

I don't see a noticeable difference in quality, so I just use GPU.

Answer 66 · 2023-05-05T00:09:45.000Z

Does anyone know how to make this work on an Apple Silicon Mac with Videotoolbox?

Answer 67 · 2023-05-05T08:54:26.000Z

Is it common for file size to be reduced drastically like a 40gb 1080p remuc of a silent voice went to 6gb when its converted to 4K

Answer 68 · 2023-05-05T08:59:27.000Z

Is it common for file size to be reduced drastically like a 40gb 1080p remuc of a silent voice went to 6gb when its converted to 4K

Completely depends on the encoder and encoder settings, but this is definitely possible.
Personally I use NVENC H264 for compatibility with a variable bitrate target of 25000Kbps, and even that is probably overkill. With AV1 encoding I can compress videos to much lower bitrates with very similar VMAF scores.

NVENC on a 1650 uses the older architecture (cant remember the names) and isn't as good as 20 series+, but use whatever produces a video where you don't notice obvious artifacts
It's much faster than CPU encoding still (although file size will be larger)

Answer 69 · 2023-05-05T09:11:24.000Z

It's much faster than CPU encoding still (although file size will be larger)

Yea it went speed 0.2x took a while left it on overnight.
I used the default settings that were given it gave such huge storage savings without any drop in quality it looks even better imo. But ill stick to nvenc because the main reason i tried anime4k instead of anything else was its speed.

Answer 70 · 2023-05-05T11:42:14.000Z

Does anyone know how to make this work on an Apple Silicon Mac with Videotoolbox?

I don't have an apple silicon Mac to try so I can only suggest options based on documentation that I have read.

Try this for h264
ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -c:v h264_videotoolbox -q:v 50 "output.mkv"

Or maybe this for h265
ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -c:v hevc_videotoolbox -q:v 50 "output.mkv"

The 50 can be 0 - 100 where 0 is the worst and 100 is the best. Try out different numbers with small video clips and compare quality to find the best number for your use case.

Answer 71 · 2023-05-05T19:00:05.000Z

@ryandash thanks for the suggestions!

Unfortunately, I don't think Apple Silicon Mac supports vulkan.

This is the error I got:

-vsync is deprecated. Use -fps_mode
Passing a number to -vsync is deprecated, use a string argument as described in the manual.
Device creation failed: -12.
Failed to set value 'vulkan' for option 'init_hw_device': Cannot allocate memory
Error parsing global options: Cannot allocate memory

ffmpeg -hwaccels:

Hardware acceleration methods:
videotoolbox

Is it possible to apply the shaders without using vulkan?

Edit: I've also tried -init_hw_device videotoolbox, but that also throws me an error.

[AVFilterGraph @ 0x600000a50360] No such filter: 'libplacebo'
Error reinitializing filters!
Failed to inject frame into filter network: Filter not found
Error while processing the decoded data for stream #0:0
[libvorbis @ 0x134e11a10] 35 frames left in the queue on closing

Answer 72 · 2023-05-05T19:12:57.000Z

@Tama47 If you use MPV instead of ffmpeg, OpenGL rendering is an option (provided it's compiled with the required libraries). But I don't know if it'll work on your setup.

Answer 73 · 2023-05-06T23:58:48.000Z

Can anyone help me?
Im using this command, the video looks great. But there seems to be some delay in the audio, around 3-4 seconds. Tried with multiple files.

.\ffmpeg.exe -y -vsync 0 -i "a.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=7680:h=4320:upscaler=ewa_lanczos:custom_shader_path=shaders/ac.glsl,hwdownload,format=yuv420p10 -c:v hevc_nvenc -c:a copy -cq 24 -bf 5 -refs 5 -preset p7 "a.mkv"

Answer 74 · 2023-05-07T00:44:59.000Z

Can anyone help me? Im using this command, the video looks great. But there seems to be some delay in the audio, around 3-4 seconds. Tried with multiple files.

.\ffmpeg.exe -y -vsync 0 -i "a.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=7680:h=4320:upscaler=ewa_lanczos:custom_shader_path=shaders/ac.glsl,hwdownload,format=yuv420p10 -c:v hevc_nvenc -c:a copy -cq 24 -bf 5 -refs 5 -preset p7 "a.mkv"

Do you have the same issues with just w=3840:h=2160?

I do not know what device you are using for playback but can the device play other local 8k content?

8k can be pretty demanding especially because you are encoding your video with h.265 and not h.264.
If you are watching the video on a device where you can monitor resource usage, how is your CPU and GPU usage?

Are you using Bluetooth audio? IDK what player you are using but depending on the device and player, bluetooth audio sync could be an issue. Reconnecting the Bluetooth device may fix that.

Answer 75 · 2023-05-07T01:32:01.000Z

@FarisR99 What would the commands be for mpv? I would also like to give it a try.

Answer 76 · 2023-05-07T10:34:27.000Z

@FarisR99 What would the commands be for mpv? I would also like to give it a try.

@Tama47 I forgot that the OpenGL implementation was only added as part of the MR adding support for vf=gpu to Windows, followed by my own commit to allow selecting the api as an argument

My fork of mpv is public which has these changes merged into master, but a few additional tweaks like respecting --vulkan-device choice when using vulkan with vf=gpu. Clone it and revert the latest 2 commits (which makes vulkan support a requirement), then build that with the required EGL libraries. Run mpv --vf=gpu=api=help and ensure egl is listed - if so you should be able to use --vf=gpu=w=3840:h=2160:api=egl.

Answer 77 · 2023-05-07T10:34:43.000Z

I got a question to ask does a higher cq value mean more compressed or less compressed and does it mean that your getting higher quality?
In this following code

ffmpeg -y -vsync 0 -i "input.mkv" -init_hw_device vulkan -vf format=yuv420p10,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -c:v hevc_nvenc -cq 24 -bf 5 -refs 5 -preset p7 "output.mkv"

And what about cpu encoding I don't see a way to change the values on that script?

Answer 78 · 2023-10-09T20:38:25.000Z

Hello everyone,
I created GUI application which allows to save upscaled anime to disk using Anime4K shaders, it internally uses ffmpeg
https://github.com/mikigal/Anime4K-GUI

Answer 79 · 2023-10-09T22:25:14.000Z

Please stop, I don't want to see random video files in the wild ruined by your meme sharpening filter

Answer 80 · 2023-11-04T02:00:51.000Z

Hello everyone, I created GUI application which allows to save upscaled anime to disk using Anime4K shaders, it internally uses ffmpeg https://github.com/mikigal/Anime4K-GUI

Thank you, you're really a lifesaver

Answer 81 · 2023-11-15T04:32:54.000Z

I'm trying with ffmpeg but I'm crashing with out of memory error, my GPU is an 1650 with 4GB of VRAM
ffmpeg.exe -y -i %%i -init_hw_device vulkan -vf format=yuv420p,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=%SHADER%,hwdownload,format=yuv420p -cq 19 -c:v hevc_nvenc "%%~dpni_out.mkv"
any idea of what can be wrong?

Answer 82 · 2023-11-15T08:16:41.000Z

I'm trying with ffmpeg but I'm crashing with out of memory error, my GPU is an 1650 with 4GB of VRAM ffmpeg.exe -y -i %%i -init_hw_device vulkan -vf format=yuv420p,hwupload,libplacebo=w=3840:h=2160:upscaler=ewa_lanczos:custom_shader_path=%SHADER%,hwdownload,format=yuv420p -cq 19 -c:v hevc_nvenc "%%~dpni_out.mkv" any idea of what can be wrong?

Ffmpeg runs out of VRAM on my 12GB 6700XT even without shaders so I think it's just an Ffmpeg/libplacebo bug, probably need 16GB VRAM or something to not crash (some files worked fine though).

Answer 83 · 2023-11-15T12:54:33.000Z

I see, someone reported the problem? I'm not sure if I have the right to report memory issue with a 4GB card

Answer 84 · 2023-11-15T14:37:48.000Z

I created an ticket in ffmpeg bug report site: https://trac.ffmpeg.org/ticket/10665

Answer 85 · 2023-11-15T15:25:32.000Z

I created an ticket in ffmpeg bug report site: https://trac.ffmpeg.org/ticket/10665

I don't have an account there but this is what it looks like for me: https://i.imgur.com/nDQ1DQe.png Doing just 1080p->1440p:
ffmpeg -y -i "input4.mp4" -init_hw_device vulkan -c:v libx264 -preset fast -crf 28 -c:a copy -c:s mov_text -metadata:s:s:0 language=English -disposition:s:s:0 forced -vf format=yuv420p,hwupload,libplacebo=w=2560:h=1440:upscaler=ewa_lanczos,hwdownload,format=yuv420p "Cells at Work! - S1E4 - Food Poisoning - 4.mp4"

Answer 86 · 2023-11-15T16:20:10.000Z

I added your print as proof, thx, should make the problem more clear since some people just say in nowdays to buy a new card when they know you has 4gb only, lol.

Answer 87 · 2023-11-15T20:26:34.000Z

Lookin' good.