*-CHACHA20-POLY1305 not recognised
osirisinferi opened this issue · 49 comments
My Apache uses ECDHE-ECDSA-CHACHA20-POLY1305 for most connections, but for some reason I can't understand, cipherscan doesn't list it in its output. The local OpenSSL does have the CHACHA20-POLY1305 ciphers in the output of openssl ciphers -v 'ALL:COMPLEMENTOFALL:+aRSA', so you'd think cipherscan would test for it?
it does, the problem is that there are about 6 different, mutually incompatible versions of the same chacha20-poly1305 ciphers.
The ones in the included openssl come from old agl draft (https://tools.ietf.org/html/draft-agl-tls-chacha20poly1305-04 I think). While the new openssl includes ones from IETF accepted draft from Nikos (https://tools.ietf.org/html/draft-ietf-tls-chacha20-poly1305-04). And if you're using GnuTLS then you may be using older version of the same draft (https://tools.ietf.org/html/draft-ietf-tls-chacha20-poly1305-00)
My server and client are using the exact same version of OpenSSL with exactly the same CloudFlare patch to add Chacha20-Poly1305. So you'd think it should detect it.
but did you tell cipherscan to use the openssl with the exact same patch? It uses the embedded one by default.
I immediately deleted those extra binaries. I'm not very fond of those unknown binaries, so got rid of them even before I ran cipherscan. From earlier experience, I knew it would use the systems OpenSSL.
Then I don't see why it would ignore it....
can you connect to the server using openssl s_client? do you need some extra options to do?
No extra options needed, although my server defaults to ~AES256-GCM-SHA384 in stead of Chacha20-Poly1305. (Note: ChaCha ciphers are entered befóre the AES ciphers and I've got SSLHonorCipherOrder on On.) Probably because of that fancy CloudFlare algorithm which detects AES-NI at the client..
yeah, if the server doesn't do a straight-up cipher order, the results may not be reliable
do you know of some servers that do use it and have chacha enabled?
blog.cloudflare.com for example. My Chromium uses Chacha/Poly, but cipherscan (and OpenSSL without -cipher) doesn't detect it.
The reason openssl s_client isn't using ChaCha by default is because of the ordering of the cipher suits in the ClientHello: OpenSSL puts the ChaCha-ciphers below all the ephemeral AES cipher suits, so the server "detects" AES-NI capabilities (it uses the client ordering as a detection tool which server-side cipher suit it should choose from the servers list of ciphers). For example, Chrome puts ChaCha at the top of the cipher suit list, so the server uses ChaCha in stead of AES.
But if you explicitly ask for ChaCha with -cipher or put it on top of the requested ciphers, it chooses ChaCha.
See the CloudFlare blog post (§Performance to be exact): Do the ChaCha: better mobile performance with cryptography for more info about this.
can't reproduce with blog.cloudflare.com, the 0xcc14 (agl draft chacha20-poly1305 cipher) is negotiated and reported as supported
and ssllabs reports that it supports both it and the new ietf version of the cipher: https://www.ssllabs.com/ssltest/analyze.html?d=blog.cloudflare.com&s=198.41.214.68
it does not detect the RSA versions of the ciphers though...
My OpenSSL can connect with the IETF as wel as the draft version of the ChaCha-cipher. I'm using this patch for OpenSSL: https://github.com/cloudflare/sslconfig
The fact it doesn't detect 0xcca9 in your case might also suggest your OpenSSL is """outdated"""? (Does the official OpenSSL already have ChaCha-support? A mailing list thread suggest it doesn't [but it's a year old] and the ChaCha-code is only implemented in OpenSSL masters branch, not in 1.0.2.)
In any case, I think there might be some tuning required with regard to ChaCha20/Poly1305 😛
the problem is, that the new openssl doesn't support all the ciphers that old versions do, so if you use old OpenSSL you won't detect if the server has weak or obsolete ciphers enabled
but we are rather well aware of it
The reason openssl s_client isn't using ChaCha by default is because of the ordering of the cipher
suits in the ClientHello: OpenSSL puts the ChaCha-ciphers below all the ephemeral AES cipher suits,
aah, so the cloudflare patches use different ordering that what the old OpenSSL did; old does put Chacha20 first
try to change line 97 from
CIPHERSUITE="ALL:COMPLEMENTOFALL:+aRSA"to
CIPHERSUITE="CHACHA20:ALL:COMPLEMENTOFALL:+aRSA"this should workaround it
It does indeed.
But, for my understanding, shouldn't cipherscan test all ciphers individually or something? Because as you say, it's a workaround 😛
The ordering of the local cipher suits shouldn't matter in detecting all possible ciphers on the server, right?
in theory it shouldn't, but first, we have servers like youtube or cloudflare which enable certain ciphers only if some other ciphers were advertised by client and secondly, using such "all ciphers minus ones already selected by server" algorithm makes testing faster, in many cases significantly so
@tomato42 rereading this thread, I'm sort of wondering what the right solution is. Should we ask @PeterMosmans to update his openssl branch with the cloudflare patch, so we support both versions of chacha20?
Hi @jvehent - it's on my TODO list (for quite a while now) to implement the new CHACHA ciphers as well. I'd like to use the official OpenSSL code for that and backport those to the 1.0.2 branch, and rather not add another (non-working) patch.
@drwetter : https://github.com/cloudflare/sslconfig
@PeterMosmans : Anything I can do to help? Where should I sent the bribing beers to? 🍺
@tomato42 rereading this thread, I'm sort of wondering what the right solution is.
first, adding all three (ietf-draft-00, RFC and agl-draft) to openssl is a bit crazy, and even if we do it, it's not really maintainable
secondly, to test if the server supports given cipher, we don't need to be able to actually negotiate it (as in "encrypt using it")
so MNSHO is that the only way forward is to extend the cscan.py to be able to perform those tests (and also CECPQ1, obsolete and current TLS1.3 drafts, Camellia-GCM, ...)
I would be fine only supporting the RFC version and begin pretending that the others don't exist.
And how do you then detect that a given server that should be updated, isn't and is still running that obsolete version of GnuTLS?
It's just like with SSLv2, just because we ignore it, doesn't mean it isn't there.
I would be fine only supporting the RFC version and begin pretending that the others don't exist.
That's acceptable to me as well.
I feel reasonably confident that there are very few servers that are:
a) proactive enough to compile and enable ChaCha20-Poly1305 before it became an IETF standard, and
b) so unmaintained that they will never turn it off
Further, there are relatively few differences between the initial agl version and the IETF version and it seems extremely unlikely that there will be any issues discovered that affect the agl version (and only the agl version) long before any of those servers have been recycled for scrap metal.
If we don't have to actually operate the cipher, but just have to say "hey, do you support cipher 0x20 0x31" and get a yes/no back, then that would be sufficient to test for old versions?
I don't think s_client supports that, or at least I've never seen it.
Is there a way to hand-craft a single cipher negotiation to the remote server for the non-RFC cipher versions, just far enough to see if the server responds "sure, I can do that" or not, without actually building an entire OpenSSL patch tree?
yes, but not with openssl s_client
If we don't have to actually operate the cipher, but just have to say "hey, do you support cipher 0x20 0x31" and get a yes/no back, then that would be sufficient to test for old versions?
yes, that's exactly what needs to be done - create a Client Hello with just old cipher versions and see if the server replies with handshake_failure, if it did send alert, it's OK, if it did send Server Hello it at least thinks it supports that old ciphers, which is bad
I feel reasonably confident that there are very few servers [like that]
Me too, but they will happen (probably already did) and if cipherscan is to be truthful, it needs to report that.
Just to be clear - I have nothing against changing the bundled openssl ChaCha20 implementation to the IETF one, but I don't think we can consider the whole issue "done and closed" after we do it.
@PeterMosmans: ohai. did you ever get a chance to patch your openssl distribution?
Hi @jvehent - sorry I didnt' get back to you earlier. Actually, work is already being done on a " new " codebase (the 1.1.1 branch). See https://www.onwebsecurity.com/announcements/rebase-openssl-1-0-2-chacha-to-use-tls-1-3.html for some background info.
The TL;DR: Still a work in progress 😉
Of course I'm biased given I'm the primary author of tlslite-ng, and for @PeterMosmans those changes to 1.1.1 will most likely not be insurmountable as that code base is fairly familiar, but I do wonder if actually creating an openssl s_client interface on top of tlslite-ng wouldn't be a better path forward, requiring less work long term...
while tlslite-ng code is missing some features of 1.0.2 (like ECDSA or full EXP support) it's more related to limited number of people working on it, not it being unwelcome - case in point, it does have both SSLv2 and TLS 1.3 support and it's going to stay there exactly to be able to verify that the server does not implement stuff like SSLv2 (or has specific SSLv2 ciphers disabled); it's also python, so it's easier to move around (ARM machines anyone?) and modify/update.
</2cent>
that unfortunately wouldn't work for testssl.sh
this is a cipherscan bug and cipherscan requirement...
It should run on every platform without installing anything else.
that's already not the case for cipherscan, for OSX you need to install bash from brew, for BSD you need to compile specific version of OpenSSL yourself, windows is supported only through cygwin, so you need to install quite a bit of stuff already
only on modern x86_64 Linux does cipherscan run out-of-box, and most Linux systems already have at least Python 2.6 or 3.2 installed...
Most of our checks incl. e.g. ROBOT, ticketbleed and checking for TLS 1.3 drafts supported are done in bash sockets.
while basics indeed can be checked using bash sockets, as soon as you need to perform real crypto (to for example collect ticket hints in TLS 1.2, renegotiation tests, resumption tests or anything but cipher and group negotiation in TLS 1.3) that's not usable solution
this is a cipherscan bug and cipherscan requirement...
You're right. My apologies! I should have taken offline this for further discussion. My point was simply that we share a common denominator and I wanted to share my project's view after your suggestion.
Cheers, Dirk
PS: If you feel like you need binaries from Peters fork just help yourself from my project.
not need to apologise 😃 I just thought you mixed up issues/projects
I have no problem discussing it, but for this particular issue it does feel rather off-topic
wanted to share my project's view after your suggestion.
fair enough
(that being said, I still don't understand why a drop-in replacement of openssl s_client wouldn't work for testssl.sh)
I just thought you mixed up issues/projects
partly. yes. That came via mail and I missed where the discussion took place. I discussed this topic @ @PeterMosmans repo, mine and via pm.
(that being said, I still don't understand why a drop-in replacement of openssl s_client wouldn't work for testssl.sh)
As you asked for... :-)
Currently I am happy that testssl.sh doesn't force a user to install anything in 99% of all cases.. It even works with the old bash 3.2.57 under Mac OS X. A "git pull" or taking a zip from the release is all it needs to run it (including binaries for several platforms). It's not for Linux only (incl. ARM7l), all BSDs, OSX and more.
I do realize from the technical perspective your idea sounds good -- testssl.sh would also need x509, enc, pkey off the top off my head. But also if that would be handled by tlslite-ng I then would have to sacrifice the ease of installation for non-Linux users and/or "part-time noobs", as far as their platform of choice is concerned.
I like the idea of a drop-in replacement to openssl s_client, but my experience shipping Python to end-users is that it's a very hard to get solid cross-platform deployments that work on all versions of linux and macos. Do you currently test tlslite-ng across a matrix of systems?
practically yes, I test on python 2.6, 2.7, 3.2 up to 3.7 and I use it in "production" on RHEL 5, 6, 7[1] and Fedora (so given that Travis uses Ubuntu, and I'm using RPM-based distros, that should cover most of the differences)
while I haven't actually ran it on macosx(tlsfuzzer/tlslite-ng#73), I'm not doing anything platform specific (it should run on Windows(tlsfuzzer/tlslite-ng#70), let alone OS X) so if there are any problems, they should be minor (and I'm willing to help, but I'm completely unfamiliar with dev environments on those platforms so help on that front would be appreciated)
in general, the code is runnable in place (there is no need to install/compile tlslite-ng or any of its dependencies to use it) so I see creating a zip file that has an openssl exectutable and few folders in it that can behave as openssl s_client when ran with a simple ./openssl on any platform (but Windows) as entirely doable
Running tlslite-ng in place is exactly what I do when describing how to run tlsfuzzer vulnerability test cases and I haven't received a single report that it doesn't work.
1 - given that we test RHEL (collectively) on i386, x86_64, ppc, ppc64, ppc64le, s390, s390x, aarch64 and ia64, and I've ran tlsfuzzer test cases on all of them, I'm quite sure about my cross-architecture support 😃
No disagreement, it all sounds reasonable to me, but you don't know for sure until you actually run it on those platform, do you?
I've been bitten enough time by the complexity of shipping code to heterogenous configuration, and those issues are hard enough to debug, that I think we should only take this route if we have solid proof that it'll work for everyone. TravisCI can help, and I could probably instrument some of this on Mozilla's Taskcluster to benefit from our large range of supported operating systems.
1 - given that we test RHEL (collectively) on i386, x86_64, ppc, ppc64, ppc64le, s390, s390x, aarch64 and ia64, and I've ran tlsfuzzer test cases on all of them, I'm quite sure about my cross-architecture support smiley
To be clear: I'm not concerned about cpu support as much as distributions and linux vs macos. Even within a given distribution, different releases often have different libraries that breaks things in unexpected ways.
Of course, we won't know until we try.
I'd say verifying that it runs on osx should be next step (before Mozilla Taskcluster), unfortunately I'm now swamped with TLS 1.3 work so can't drive this project myself.
different releases often have different libraries that breaks things in unexpected ways.
sure, but given that we can bring all our libraries with ourselves, the only thing we really depend on is python itself
I'm in favor. Also, I have a gsoc project to rewrite cipherscan, and this is something we could try then.
@tomato42 I'm more than happy to help you with an openssl stub for tlslite-ng (time permitting, of course) - given, that it makes sense (see @jvehent 's remarks) to pursue this path.
Maintaining the insecure openssl patches becomes more and more time consuming, given that especially the insecure parts are aggressively being yanked from the official master repo. Using a more independent version sounds like it make more sense - however, there's the slight chance that the openssl stub layer might actually introduce compatibility issues or false positives/negatives.
Would it make sense to declare beforehand which distro's / architectures are going to be supported ? Should any and all distro('s) be supported by cipherscan ?
ok, that means no openssl activities anymore, @PeterMosmans?
Would it make sense to declare beforehand which distro's / architectures are going to be supported ? Should any and all distro('s) be supported by cipherscan ?
Unfortunately, the users dictate which architectures should be supported :)
That said, I think we could reasonably narrow it down to the last 2 MacOS version, and rhel/ubuntu/arch, then tell unsupported systems to run it in a container.
If we simply say that the toolchain has to be built with brew on macOS with a reasonably defined requirements.txt, that should do a good job of decoupling it from the macOS version anyways. I've never had any problems with Python crypto libraries in brew.
Hi @drwetter - no, it certainly doesn't mean that, I'll still be working on the insecuressl / openssl fork. Don't worry 😉
I do like the idea though from @tomato42 of crafting an openssl-like interface on top of an 'independent' SSL/TLS Python implementation (tlslite-ng). That can be maintained without the burden of having official maintainers removing (insecure) functionality. In an ideal world, both cipherscan and testssl.sh can use the same insecure openssl (-compatible) binaries of course. Time is the biggest constraint here, and I want to maximize the 'return on investment'.
Ideally, we would do both a light handshake and a full one. There's value in doing a full handshake and sending data through the transport to detect implementation issues. That means we should continue using openssl regardless of this plan, but cover a larger surface with tlslite-ng.
@PeterMosmans: good! :-)
I never saw maintaining as a big issue. I would be completely happy with more a one time thing: getting a new fork of openssl "properly" downgraded, security-wise. Don't know what @tomato42's and @jvehent 's requirement is -- I would be only looking for bad features in your fork (I assume that the advanced features come with the vanilla version). That means I never cared whether the version is 1.X.Y-c or 1.X.Y-m, supposed -m doesn't have bugfixes or features which testssl.sh is depending on or at least would be helpful.
there's the slight chance that the openssl stub layer might actually introduce compatibility issues or false positives/negatives.
there's always possibility that software doesn't behave in ways we think it does... same can be said about those rebases/backports...
e.g sending SSLv2-style Client Hello using OpenSSL with TLS1.3 support definitely requires some extra magic on cipherscan script level
I would be only looking for bad features in your fork (I assume that the advanced features come with the vanilla version)
I was thinking of doing everything but certificate verification on tlslite-ng level (the fake openssl executable)
not sure what @jvehent had in mind with "That means we should continue using openssl regardless of this plan, but cover a larger surface with tlslite-ng." Was it the cscan.py util?