upgrade to OpenSSL 3.0.0 causes stuttering audio
Closed this issue · 5 comments
The new OpenSSL 3.0.0 (with GPLv3 compatible Apache license) makes some changes in libcrypto
that disagree with RPiPlay:
After getting OpenSSL 3.0.0 by installing Debian unstable packages libssl3 + libssl-dev + openssl on ubuntu 20.04,
RPiPlay builds without problem, and ldd rpiplay shows libcrypto.so.3 (instead of libcrypto.so.1.1 in an openssl-1.1.1x build).
But streamed music has audio stuttering with the openssl3.0.0 build that is not there with the 1.1.1 build, so libcrypto changes are breaking something. This is on linux desktop, with gstreamer rendering, but since libcrypto is only used in the airplay raop library, it shouldn't depend on the renderer. Using OpenSSL 3.00 built from source behaves identically.
It would be great if anyone here who is a libcrypto expert could find what the issue is. The openssl3 release notes
state that some things in applications may break and need to be fixed after the upgrade.
OpenSSL3 is the future, as it removes the license obstacle to distribution of compiled GPLv3 code linked to OpenSSL.
(I am interested in the change, because someone wants to package UxPlay 1.38 (which I maintain) for debian, which is only possible with OpenSSL3, but I am no libcrypto expert)
OpenSSL 1.1.1 will be maintained for another 2 years only.
Some more clues to this.
(1) The problem must be in lib/crypto.c where the only direct link to openssl is made.
(2) It must be in the "common AES Utilities" used by raop_buffer.c and mirror_buffer.c
which are are used for decrypting the streamed audio and video (as opposed to the ellitptic curve crypto
which is used for the initial pairing between server and client.)
(3) more evidence for this is that the original antimof version of the RPiPlay fork uxplay http://github.com/antimof/UxPlay has exactly the same problem with OpenSSL 3.0.0, but it uses an older version of the RPiPlay crypto.c where the elliptic curve stuff is not in crypto.c, and not handled by openssl. In this case basically the only things in crypto.c are the "common AES Utilities".
Maybe the best strategy to find what needs to be fixed might be to get the antimof uxplay (or an older version of RPiPlay) working with OpenSSL 3.0.0, and then port the fix to current RPiPlay and UxPlay.
(4) Its not obvious from the OpenSSL 3.0.0 migration guide
https://www.openssl.org/docs/man3.0/man7/migration_guide.html
where the problem might be.
I have examined the code called by RPiPlay in OpenSSl 3.0.0 in detail (at least the code called to decrypt the audio stream);
there are no obvious code changes in anything called by RPiPlay (just options to use new features that dont seen to be triggered by the way the EVP functions EVP_EncryptUpdate and EVP_DecryptUpdate are called from crypto.c)
I saw that crypto.c handles the AES_CTR video decryption using EVP_EncryptUpdate not EVP_DecryptUpdate, and thought that might be some kind of bug, but it is just using the symmetric nature of the Encryption <-> Decryption algorithm. "fixing" the code to use EVP_Decrypt_Update for both audio (AES_CBC) and video (AES_CTR) made no difference to the problem. (the fix just makes the code look more logical but doesn't affect anything).
So it's a mystery why audio quality seems to get affected by using OpenSSL 3.0.0 instead of 1.1.1. Maybe it's to do with compiler options that make something run slower??
Does adding -DOPENSSL_NO_AES_CONST_TIME to a new build of OpenSSL make a difference?
From the OpenSSL commit message for this:
This seems to require rebuilding OpenSSL 3.0.0 with "no-asm"
I'll check it out
This adds optional constant time support for AES
when building openssl for no-asm.
Enable with: ./config no-asm -DOPENSSL_AES_CONST_TIME
Disable with: ./config no-asm -DOPENSSL_NO_AES_CONST_TIME
This is by default enabled.
The OpenSSL3 INSTALL.md says
no-asm
Do not use assembler code.
This should be viewed as debugging/troubleshooting option rather than for production use. On some platforms a small amount of assembler
code may still be used even with this option.
Fixing the bug in codec.c reported as #283 with a small workaround in lib/buffer.c fixed the problem with OpenSSL3
For AES_CTR, the OpenSSl; decrypter EVP_DecryptUpdate really needs to be followed by EVP_DecryptFinal
but the simple workaround of calling aes_ctr_decrypt with a buffer size of encryptlen + 15 instead of encryptlen makes OpenSSL do its work properly. Unfortunately FD--'s crypto.c implementation zeroed out the last 16 encrypted bytes without this fix.
A by-product of the fix is that has become trivial to add back support for ALAC (as in shairplay) to RPiPlay.