libfdk_aac is not legally redistributable
Artoria2e5 opened this issue · 8 comments
Describe The Bug:
libfdk_aac is not legally redistributable with ffmpeg.
To Reproduce:
Run /usr/local/lib/node_modules/homebridge-camera-ui/node_modules/ffmpeg-for-homebridge/ffmpeg -L, or wherever the module is found.
Expected behavior:
Something without the last two lines:
ffmpeg version 5.0.1-5.0.1 Copyright (c) 2000-2022 the FFmpeg developers
built with Apple clang version 11.0.3 (clang-1103.0.32.62)
configuration: --enable-videotoolbox --enable-nonfree --enable-gpl --enable-openssl --enable-libsvtav1 --enable-libx264 --enable-libx265 --enable-libvpx --enable-libxvid --enable-libvidstab --enable-libaom --enable-libzimg --enable-lv2 --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libtheora --enable-libfdk-aac --enable-libsrt --enable-libspeex --disable-debug --disable-doc --disable-shared --enable-pthreads --enable-static --enable-small --enable-version3 --extra-cflags='-I/Users/Shared/ffmpeg-for-homebridge/workspace/include -I/Users/Shared/ffmpeg-for-homebridge/workspace/include/lilv-0' --extra-ldexeflags= --extra-ldflags=-L/Users/Shared/ffmpeg-for-homebridge/workspace/lib --extra-libs='-ldl -lpthread -lm -lz' --pkgconfigdir=/Users/Shared/ffmpeg-for-homebridge/workspace/lib/pkgconfig --pkg-config-flags=--static --prefix=/Users/Shared/ffmpeg-for-homebridge/workspace --extra-version=5.0.1
libavutil 57. 17.100 / 57. 17.100
libavcodec 59. 18.100 / 59. 18.100
libavformat 59. 16.100 / 59. 16.100
libavdevice 59. 4.100 / 59. 4.100
libavfilter 8. 24.100 / 8. 24.100
libswscale 6. 4.100 / 6. 4.100
libswresample 4. 3.100 / 4. 3.100
libpostproc 56. 3.100 / 56. 3.100
This version of ffmpeg has nonfree parts compiled in.
Therefore it is not legally redistributable.
Screenshots:
Use above paste.
Environment:
- Operating System: macOS, but does it really matter when the whole shtick of this project is fdk?
2. Redistribution and use in source and binary forms, with or without modification, are permitted without payment of copyright license fees.
@Artoria2e5 Can you please post the follow up information that you shared on Reddit.
Ah well, actually the stuff has been followed-up once in the deleted #56. But anyways.
Legality
The problem here is the combination --enable-nonfree --enable-gpl. While FDK is perfectly happy to be distributed on its own, its GPL incompatibility means that they can't be linked together and then distribtued. See:
- https://github.com/HandBrake/HandBrake/blob/master/LICENSE, about why handbrake used to do this but stopped
- https://ffmpeg.org/legal.html, about the reasoning (GPL applies to whole product but GPL cannot be applied on incompatibly licensed code -> kaput)
Workarounds
Since we have two parts that don't mix we need to think about what these two parts are and what they do.
- fdk_aac is our aac encoder. It makes AAC-ELD, which is one of HAP's streaming formats, and regular AAC of nice quality.
- The most important GPL part is x264, which we use on every system other than Rasp Pi.
If we want to keep fdk_aac, we can do that by building without --enable-gpl. But that means to get H.264 we'd use --enable-openh264, which is not the best encoder. We are losing a lot of video compression to get audio compression, and we all know that video makes up much more of the bandwidth than audio.
So that leaves us with ditching fdk_aac. Can we do that? Yes.
- On macOS, the ffmpeg "aac" encoder delegates to system AudioToolbox, which is quite good and can make AAC-ELD.
- On other systems, ffmpeg uses its own aac encoder and the above won't fly. However, HAP supports Opus as a streaming codec type, and according to this comment it works. Recordings will still need to use AAC, since the two choices are AAC-LC and AAC-ELD. The former is going to work just fine, since the ffmpeg encoder makes AAC-LC.
What would be the consequence here?
- On macOS, not much. Back when Apple's code was available as "QAAC", this stuff was loved.
- On other systems with the changes: Opus should give comparable quality. FFmpeg AAC might fall short, but we aren't doing audiophile streams over homekit.
- Recording will no longer be able to share a AAC-ELD stream. Curiously, camera-ui doesn't do any sharing, but encodes a separate AAC-LC stream here. But anyways, audio encoding is cheap.
I’m not going to weigh in on any legal points, I’m not a lawyer, but I can weigh in on proposed solutions.
On macOS, the ffmpeg "aac" encoder delegates to system AudioToolbox, which is quite good and can make AAC-ELD.
In my experience, the number of users on macOS is effectively a rounding error, so that barely helps us.
On other systems, ffmpeg uses its own aac encoder and the above won't fly. However, HAP supports Opus as a streaming codec type, and according to this comment it works.
Unless something has changed recently, Opus “works” but not very well, resulting in audio issues in most cases. Additionally, switching from AAC-ELD to Opus will require deleting and recreating every camera, as HomeKit seems to refuse to acknowledge the capabilities of the camera changing and continues to expect AAC-ELD.
In my experience, the number of users on macOS is effectively a rounding error, so that barely helps us.
Guess I'm the rounding error! Anyways, it makes sense. It's not really a cool system or anything.
Unless something has changed recently, Opus “works” but not very well [...] refuse to acknowledge the capabilities
Oh no. There is maybe a third approach involving some cross-process wizardry, assuming that someone wants to do all the plumbing. If we pipe something to fdkaac and mux the stuff back...
Ah, and then we hit the part where people don't even dare to distribute fdkaac regardless of GPL because of patent issues. nu774/fdkaac#43 (comment) Not that I think Fraunhofer will sue you for this, since every single codec is like that. Like H.264, which we have to use anyways.
Opus “works” but not very well
There might also be another explanation in this. In Camera-UI I see that the ELD stuff was translated into "lowdelay" for opus, which is a very bad idea because doing so disables voice modes (OPUS_APPLICATION_RESTRICTED_LOWDELAY), when all we want from surveillance is voice. The code that does that is in https://github.com/xiph/opus/blob/82ac57d9f1aaf575800cf17373348e45b7ce6c0d/src/opus_encoder.c#L1352-L1354.
I mean, how tightly do we need to squeeze Opus anyways? AAC-LD was ~20 ms compared to Opus at 26.5. AAC-ELD claims 7.5. The smallest delay Opus can use while still having speech-coding is 16.5 ms (-frame_duration 10 + 6.5 overhead).
To follow
So I think the general consensus has been that this is a gray error due to the inherent license incompatibilities. Based off the above both with experiences with OPUS that result in a "it kinda of works" state and the fact that everyone would have to re-add every camera to their HomeKit, I think we will end up sticking with using the fdkaac. While just having people compile from source is an option, it isn't viable due to the compile times especially considering the majority of people are running on RPI hardware. However, if you would like to fork and implement a solution that uses OPUS, you are more than welcome to open a PR. I know myself and some of the other more active devs do not have the time to dedicate to such a switch nor know of the ins and outs of the wizardy you were talking about to improve OPUS's quality. We can then make a decision about merging it in if the performance and quality are on par with fdkaac.
I think the license incompatibilities have essentially been accepted at this point. fdkaac#43 that you linked above has some distribution a compiled binary in it. Homebridge started off as a relatively small project that was just a PoC and is provided as non-commercial software so at least from my perspective there has been little concern in regards to minor licensing incompatibilities such as this.
I am going to close this issue for now but if you end up creating a PR you can reference this issue and we can re-open it.
I think the license incompatibilities have essentially been accepted at this point. fdkaac#43 that you linked above has some distribution a compiled binary in it.
fdkaac does not do a link to GPL, hence has no license incompatibility there. There are two layers to this (or you can say too bunch of people in some position of suing): the ffmpeg gpl-authors and the AAC patent people. fdkaac only needs to worry about the latter.
Will ffmpeg sue? Probably not. But they (used to) have a Hall of Shame for this stuff to encourage good license practice.