xp-framework/core

Consistent SSL

thekid opened this issue · 10 comments

thekid commented

Note: This is the reincarnation of xp-framework/xp-framework#209 /cc @kiesel

Using self-signed certificates and/or an internal CA yields problems with ext/ldap and and ext/curl:

$ xp -w '(new \peer\ldap\LDAPConnection("ldaps://ldap.1and1.org"))->connect()'
Uncaught exception: Exception peer.ConnectException (Cannot connect to ldaps://ldap.1and1.org:636)
  at <main>::ldap_bind() [line 101 of LDAPConnection.class.php] ldap_bind(): Unable to bind to server: Can't contact LDAP server

(With LDAP debugging, we see the reason: TLS certificate verification: Error, self signed certificate in certificate chain)

$ xp -w '(new \peer\http\HttpConnection("https://bitbucket.1and1.org"))->get()'
Uncaught exception: Exception io.IOException (60: SSL certificate problem: self signed certificate in certificate chain)
  at peer.http.CurlHttpTransport::send(peer.http.HttpRequest{}, 60, 2) [line 123 of HttpConnection.class.php]

(Here, it's obvious)

The problem is that these don't use the Windows certificate store which contains the certificate, while ext/openssl does since PHP 5.6.0 - see the Changelog:

Fallback to Windows CA cert store for peer verification if no openssl.cafile ini directive or "cafile" SSL context option specified in Windows.

Source: http://php.net/ChangeLog-5.php#5.6.0 and the code

thekid commented

For LDAP:

$ xp -w '
putenv("LDAPTLS_CACERT=C:\\tools\\cygwin\\etc\\pki\\ca-trust\\extracted\\pem\\tls-ca-bundle.pem");
return (new \peer\ldap\LDAPConnection("ldaps://ldap.1and1.org"))->connect()'
thekid commented

For CURL:

curl_setopt(
  $this->handle,
  CURLOPT_CAINFO,
  "C:\\tools\\cygwin\\etc\\pki\\ca-trust\\extracted\\pem\\tls-ca-bundle.pem"
);

...in CurlHttpTransport

thekid commented

Re:

This would need to be done by a scriptable utility though
(source)

If we're not on Cygwin, we can use the Windows Certificate store by exporting it to a bundle file programmatically. This could be done during installation or by checking LastSyncTime from HKCU\SOFTWARE\Microsoft\SystemCertificates\AuthRoot\AutoUpdate

using System;
using System.Text;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;

public class ExportWindowsCertificates
{
    private static string ExportToPEM(X509Certificate cert)
    {
        var builder = new StringBuilder();

        builder.AppendLine("-----BEGIN CERTIFICATE-----");
        builder.AppendLine(Convert.ToBase64String(
            cert.Export(X509ContentType.Cert),
            Base64FormattingOptions.InsertLineBreaks)
        );
        builder.AppendLine("-----END CERTIFICATE-----");

        return builder.ToString();
    }

    private static void ExportStore(StoreName name)
    {
        var store = new X509Store(name, StoreLocation.CurrentUser);

        store.Open(OpenFlags.OpenExistingOnly);

        Console.Error.WriteLine("From {0}", name);

        foreach (var cert in store.Certificates)
        {
            Console.Write(ExportToPEM(cert));
            Console.Error.WriteLine("- {0}", cert.Subject);
        }

        Console.Error.WriteLine("{0} certificates exported", store.Certificates.Count);
        Console.Error.WriteLine();

        store.Close();
    }

    static void Main()
    {
        ExportStore(StoreName.Root);                  // trusted root certificate authorities
        ExportStore(StoreName.AuthRoot);              // third-party certificate authorities
        ExportStore(StoreName.CertificateAuthority);  // intermediate certificate authorities
    }
}

See https://msdn.microsoft.com/en-us/library/system.security.cryptography.x509certificates.storename(v=vs.110).aspx

thekid commented

Running a webserver with a self-signed cert:

$ openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes
Generating a 2048 bit RSA private key
.............................................................................+++
....................................................+++
writing new private key to 'key.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:DE
State or Province Name (full name) [Some-State]:BW
Locality Name (eg, city) []:Karlsruhe
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Localhost
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:127.0.0.1
Email Address []:

Timm@slate ~/tmp
$ openssl s_server -key key.pem -cert cert.pem -accept 44330 -www
Using default temp DH parameters
ACCEPT

Then use the following client code:

<?php namespace https;

use util\cmd\Console;
use peer\URL;

$url= new URL($argv[1]);

$handle= curl_init();
curl_setopt($handle, CURLOPT_HEADER, 1);
curl_setopt($handle, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt($handle, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($handle, CURLOPT_SSL_VERIFYPEER, 1);
curl_setopt($handle, CURLOPT_CAINFO, $argv[2]);
curl_setopt($handle, CURLOPT_URL, $url->getURL());

$return= curl_exec($handle);

if (false === $return) {
  Console::writeLinef('%d: %s', curl_errno($handle), curl_error($handle));
  $result= 1;
} else {
  Console::writeLine($return);
  $result= 0;
}

curl_close($handle);
return $result;

....and call it as follows:

$ xp -m devel/xp/networking/ https.script.php https://127.0.0.1:44330/ \
  'C:\tools\cygwin\etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem'
60: SSL certificate problem: self signed certificate

$ xp -m devel/xp/networking/ https.script.php https://127.0.0.1:44330/ \
  'tmp/cert.pem'
HTTP/1.0 200 ok
# ...
thekid commented

The tls-ca-bundle.pem could be part of the XP runners facility. The runners would set an environment variable XP_CACERT (e.g.) to the file path of this file:

OS/Environment Paths
Linux /etc/pki/tls/certs/ca-bundle.crt, /usr/share/ssl/certs/ca-bundle.crt, /etc/ssl/certs/ca-bundle.crt, /etc/ssl/certs/ca-certificates.crt
Cygwin $(cygpath -aw /etc/pki/tls/certs/ca-bundle.crt)
Windows $LOCALAPPDATA/XP/bundle.crt, $(export-windows-certificates.exe > $LOCALAPPDATA/XP/bundle.crt)
Mac $(security find-certificate -a -p /System/Library/Keychains/SystemRootCertificates.keychain)
Other Download https://curl.haxx.se/ca/cacert.pem

Code to access it would use System::certBundle() to retrieve the path to the .pem file.

Additionally, the PEM file could be managed by a cert subcommand:

$ xp cert list
# Lists all certificates - system-wide and xp installation specific

$ xp cert trust 'self-signed/cert.pem'
# Trusts a given certificate for this xp installation

$ xp cert revoke '98:17:DD:5E:F2:73:57:D1:D3:5B:B3:FD:FF:4B:87:12:B8:A9:5F:28'
# Revoke certificate with given fingerprint for this xp installation
thekid commented

With the cert tool in place, we could run the following from the installer:

Cygwin:

Timm@slate ~/.xp
$ /path/to/cert.exe up
@cygwin (detected)
Updating certificates

> Linked ca-bundle.crt -> /etc/pki/tls/certs/ca-bundle.crt
  169 certificates

Done, C:\tools\cygwin\home\Timm\.xp\ca-bundle.crt updated

Mac OS X:

The-Mac:.xp thekid$ mono cert.exe up
@macosx (detected)
Updating certificates

> From /System/Library/Keychains/SystemRootCertificates.keychain: [.....]
  211 certificates

Done, /Users/thekid/.xp/ca-bundle.crt updated

...and similar for Linux and Unix systems.

  • For the Debian package, we'd need to do this in postinstall.
  • For plain Windows, we need to add this to setup instructions or better yet, create a one-line installer like chocolatey

The framework code would access this file by looking at the XP_CACERT environment variable.

thekid commented

The framework code would access this file by looking at the XP_CACERT environment variable.

The framework could check:

  • An SSL_CERT_FILE environment variable, see https://www.python.org/dev/peps/pep-0476/#trust-database
  • If $HOME is set: Unix User dir ($HOME/.xp/ca-bundle.crt)
  • If $LOCALAPPDATA is set: Windows user dir ($LOCALAPPDATA/Xp/ca-bundle.crt)
  • If $XDG(.+) is set:_ XDG Config dir (${XDG_CONFIG_HOME-$HOME/.config}/xp/ca-bundle.crt)
  • XP Binary file location ($(dirname $0)/ca-bundle.crt)

It would need the binary path, the others can be easily composed by checking OS environment variables,

thekid commented

Implementation TODOs

For this to work on Windows without Cygwin and Mac OS without homebrew, we should:

  • Bundle cert.exe with XP runners
  • Run cert.exe during XP runner setup
  • Release xp-runners/reference 7.8.0