Build a tiny FFmpeg

Question

Build a tiny FFmpeg

Aleksid1 opened this issue 8 months ago · 15 comments

I need to build a tiny shared FFmpeg. I removed many .sh files in /scripts.d/ directory with unused libs for my app. However it ignores my changes and build a heavy FFmpeg as with all default libs included in gpl-shared variant. What am I doing wrong?

Answer 1 · 2024-06-19T15:26:24.000Z

You probably forgot to build a new image. If you only run build.sh, it'll download the latest image from github with everything in it.

Answer 2 · 2024-06-19T17:45:59.000Z

Many thanks for your quick response and great support! You're right. I'll try tomorrow.

Answer 3 · 2024-06-19T19:01:21.000Z

If you only modified scripts, and not the core-image, you can also export QUICKBUILD=1 before making the new image. It'll download the base images from github then, and only build your local scripts.

Answer 4 · 2024-06-20T14:13:58.000Z

Thanks for the advice with export QUICKBUILD=1
I added this line to my .sh script before the makeimage command.
Is there a way to determine which lib from the /scripts.d directory is required? I removed about 30 libs to build tiny FFmpeg. And when I tried to create a new image, I was missing freetype, then libvmaf, etc.

Answer 5 · 2024-06-20T14:16:26.000Z

Nothing from there is strictly required, except for maybe the very basic mingw stuff at the lowest numbers.
There are complex inter-dependencies though, and if you remove something from a number below 50, you can expect something from a higher number to fail.
FFmpeg itself will happily build without any of it though.

Answer 6 · 2024-06-21T11:59:20.000Z

I tried to build an image for original scripts (not modified) and got this error:

=> [layer-50-libjxl 2/3] RUN --mount=src=scripts.d/50-libjxl/45-lcms2.sh,dst=/stage.sh --mount=src=.cache/dow  1043.6s
WARNING: local cache import at .cache/ghcr.io/btbn/ffmpeg-builds/win64-lgpl-shared-6.1_latest not found due to err: could not read .cache/ghcr.io/btbn/ffmpeg-builds/win64-lgpl-shared-6.1_latest/index.json: open .cache/ghcr.io/btbn/ffmpeg-builds/win64-lgpl-shared-6.1_latest/index.json: no such file or directory
ERROR: failed to receive status: rpc error: code = Unavailable desc = error reading from server: EOF
+ rm -f /tmp/tmp.lVHpIarWvU.toml

export QUICKBUILD=1
./makeimage.sh win64 lgpl-shared 6.1

FFmpeg-Builds-latest.zip was downloaded on June 13rd.

Ubuntu 24.04 on WSL2

Answer 7 · 2024-06-25T12:19:08.000Z

I tried to makeimage without any changes in your scripts. Regrettably it fails:

" => [layer-50-onevpl 1/1] RUN --mount=src=scripts.d/50-onevpl.sh,dst=/stage.sh --mount=src=.cache/downloads/50  1469.8s
 => [layer-50-openh264 1/1] RUN --mount=src=scripts.d/50-openh264.sh,dst=/stage.sh --mount=src=.cache/download  1442.7s
WARNING: local cache import at .cache/ghcr.io/btbn/ffmpeg-builds/win64-lgpl-shared-6.1_latest not found due to err: could not read .cache/ghcr.io/btbn/ffmpeg-builds/win64-lgpl-shared-6.1_latest/index.json: open .cache/ghcr.io/btbn/ffmpeg-builds/win64-lgpl-shared-6.1_latest/index.json: no such file or directory
ERROR: failed to receive status: rpc error: code = Unavailable desc = error reading from server: EOF
+ rm -f /tmp/tmp.9tO2rq6t60.toml"

docker builder prune --all
git clone https://github.com/BtbN/FFmpeg-Builds.git
cd FFmpeg-Builds
export QUICKBUILD=1
./makeimage.sh win64 lgpl-shared 6.1

WSL2 / Ubuntu 24.04

Answer 8 · 2024-06-25T12:21:03.000Z

That looks to me like a network issue or temporary outage of something.
It should not be talking to Github at that point, so probably some internal issue of Docker?
Make sure your docker installation is intact and up to date.

Answer 9 · 2024-06-25T12:48:04.000Z

Thanks for your support!
I recently downloaded Docker for WSL2 from Microsoft Store.
I'll try with native Ubuntu without WSL2.

Answer 10 · 2024-06-25T12:49:01.000Z

I never used the store version.
I usually just install docker proper on WSL these days, since that works just fine if systemd is enabled.

Answer 11 · 2024-06-26T14:47:55.000Z

I reinstalled Ubuntu 24.04 on WSL2 and installed docker from the console (not Docker Desktop). The same problem.
Is it possible to disable use of cache for docker in your script?

Answer 12 · 2024-06-26T14:49:11.000Z

It has nothing to do with any caching. The cache warning is normal if there is no cache yet.
Building works perfectly fine for me, so I have no idea what causes this for you.

Answer 13 · 2024-08-21T10:24:02.000Z

Hello. I had a similar experience and solved it.

The problem in my environment was that the memory and swapzone on the wsl side was not set properly with the .wslconfig settings.

I had previously had a wsl2 side error, so I had put a limit on memory usage in .wslconfig and had swap set to 0.

Due to this issue, during the docker image creation, cache, etc. was not created properly due to insufficient memory and the script crashed and terminated without a proper error message.

I have since reset the wslconfig settings to allow enough memory and swap and the issue was resolved.

It would be nice if you could include a warning for users on the WSL side to prevent unfortunate situations like mine.

Answer 14 · 2024-08-21T12:26:15.000Z

Thanks for your advice! I'll try again to build a custom build.

Answer 15 · 2024-08-23T14:35:49.000Z

Many thanks for your advice! I successfully created a custom build for FFmpeg.

I removed some unused libs (in scripts.d dir) to build a tiny FFmpeg custom build. My script:

export QUICKBUILD=1
./makeimage.sh win64 gpl 6.1
./build.sh win64 gpl 6.1

I created in a new file .wslconfig in C:\Users\MyUserProfile\.wslconfig

[wsl2]
memory=7GB
swap=0

Then Docker worked correctly, BtbN's scripts produced custom image and then custom build for FFmpeg.